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MONOTONE  OPTIMAL  PREVENTIVE 
MAINTENANCE  POLICIES  FOR 
STOCHASTICALLY  FAILING  EQUIPMENT 

Michael  Q.  Anderson 

Robert  O.  Anderson  Schools  of  Management 
University  of  New  Mexico 
Albuquerque,  New  Mexico 

ABSTRACT 

This  paper  examines  various  models  for  maintenance  of  a  machine  operating  subject 
to  stochastic  deterioration.  Three  alternative  models  are  presented  for  the  deterioration 
process.  For  each  model,  in  addition  to  the  replacement  decision,  the  option  exists  of 
performing  preventive  maintenance.  The  effect  of  this  maintenance  is  to  'slow*  the 
deterioration  process.  With  an  appropriate  reward  structure  imposed  on  the  processes, 
the  models  are  formulated  as  continuous  time  Markov  decision  processes,  the  optimality 
criterion  being  the  maximization  of  expected  discounted  reward  earned  over  an  infinite 
time  horizon.  For  each  model  conditions  are  presented  under  which  the  optimal  mainte¬ 
nance  policy  exhibits  the  following  monotonic  structure.  First,  there  exists  a  control  limit 
rule  for  replacement.  That  is,  there  exists  a  number  i*  such  that  if  the  state  of  machine 
deterioration  exceeds  i*  the  optimal  policy  replaces  the  machine  by  a  new  machine. 

Secondly,  prior  to  replacement  the  optimal  level  of  preventive  maintenance  is  a  nonin¬ 
creasing  function  of  the  state  of  machine  deterioration.  The  conditions  which  guarantee 
this  result  have  a  cost/benefit  interpretation. 

INTRODUCTION 

There  is  a  substantially  large  collection  of  papers  in  the  machine  repair/main¬ 
tenance/replacement  literature.  Two  fairly  comprehensive  surveys  are  McCall  [22]  and  Pier- 
skalla  and  Voelker  [231.  Specific  articles  of  interest  are  Barlow  and  Proschan  [4],  Derman  [8], 
[91,  [101,  Klein  [171,  Kolesar  [181,  Kalymon  [ISl,  Thompson  [291,  Kamien  and  Schwartz  [16], 
Ross  [251,  and  Rosenfield  [241.  An  interesting  semi-Markovian  treatment  of  shock  models  is 
given  by  Feldman  [111.  The  focus  of  these  models  has  been  on  the  replacement  decision.  In 
such  models,  gradual  operating  deterioration  (or  "wearing  ouf)  is  not  suhiect  to  control.  The 
model  in  [161  differs  somewhat  in  that  preventive  maintenance  can  be  applied  to  decrease  the 
failure  probability  over  time. 

In  this  paper  we  formulate  three  machine  maintenance  models  that  in  addition  to  the 
replacement  decision,  incorporate  the  option  of  performing  preventive  maintenance.  The  effect 
of  the  maintenance  is  to  "slow”  the  rate  of  machine  deterioration.  Each  model  presents  a 
different  version  of  the  nature  by  which  maintenance  action  alTects  the  deterioration  process. 

Results 

For  each  model  conditions  are  given  under  which  an  optimal  maintenance  policy  exhibits 
the  following  monotonic  structure: 
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1.  Control  limit  rule  for  replacement.  There  exists  0  <  i*  <  +<»  such  that  when  the  state 
of  the  machine  i  >  /*,  the  machine  is  to  be  replaced. 

2.  Decreasing  preventive  maintenance.  For  0  <  /  <  i*  denote  an  optimal  maintenance 
level  by  a*(i).  Then  a*  is  a  nonincreasing  function  of  i.  (Compare  to  [16].) 

The  technique  employed  to  obtain  the  above  results  first  involves  transforming  the  con¬ 
tinuous  time  Markov  decision  process  (CTMDP)  to  an  equivalent  discrete  time  Markov  process 
(DTMDP).  The  infinite  horizon  problem  will  then  be  approached  via  the  associated  finite  hor¬ 
izon  discrete  time  process,  thus  permitting  induction  arguments  on  the  "n-period"  return  func¬ 
tions.  We  then  use  the  fact  that  the  n-period  return  converges  to  the  infinite  horizon  return 
function  (see  Bertsekas  [5],  Denardo  [7],  Ross  [26]).  This  technique  has  been  successfully 
employed  by  others,  notably  Lippman  [19),  [201  in  queueing  optimization. 

Papers  which  focus  on  other  aspects  of  optimal  control  of  the  machine  maintenance/ repair 
problem  are  Albright  [1],  Crabill  [6],  Goheen  (12),  and  Winston  [32],  [33]. 

MODEL  DESCRIPTION -MODEL  1 

We  consider  a  machine  operating  over  an  infinite  time  horizon.  The  state  of  the  machine 
is  represented  by  /  €  S  =  Sq  U  {/),  where  So=  {0,1,2,  ...).  State  zero  represents  a  new 
machine,  state  /  a  failed  machine,  and  increasing  integer  state  values  represent  increasing  stages 
of  machine  deterioration. 

The  action  space  is  denoted  by  A  and  consists  of  triples  (oi,  02,  oj)  €  and  is  defined 
as  follows:  03 »  0  or  1  according  to  whether  the  machine  is  not  or  is  replaced  by  a  new 
machine.  If  a  machine  in  state  i  is  replaced  by  a  new  machine,  the  transition  i  — '  0  takes  place 
instantaneously.  When  03  “  0,  the  transition  1  — •  /  +  1  occurs  at  (exponential)  rate  X|(/,  0|) 
and  the  transition  to  failure  i  -*  f  occurs  at  rate  X/(i,  02).  The  term  "rate"  is  used  here  in  the 
usual  sense,  see  Ross  [26].  For  example,  let  p^,+i  (ai)  be  the  transition  probability  for  the 
transition  1  — *  /  +  1  given  action  O)  and  suppose  the  transition  time  probability  out  of  state  i  is 
exponential  with  mean  l/\((.  Aj).  Then  X|(/,  O])  =*  K(i,  ai)p,,/+i  (aj).  Finally,  if  the  machine 
is  in  the  failed  state,  it  must  be  immediately  replaced  (aj  -  1  when  i  —  f).  a^  and  02  ar® 
viewed  as  maintenance  actions  in  a  sense  to  be  made  precise  below. 

The  reward  structure  is  specified  as  follows.  A  lump  sum  cost  C  is  incurred  when  the 
machine  is  replaced  by  a  new  machine.  If  the  machine  fails  while  in  service  a  lump  sum  cost 
f  ^  C  is  incurred  for  replacement.  If  the  state  is  1  €  So,  the  machine  earns  revenue  at  rate 
r(i,  0)  per  unit  time,  0  »  (01,  02.  0).  The  optimality  criterion  is  the  maximization  of 
expected  discounted  revenue  earned  over  an  infinite  time  horizon.  For  the  continuous 
discounting  of  cash  flows  denote  the  discount  factor  by  a  €  (0,  1). 

flotation  and  Definitions.  For  a  real-valued  function  g  defined  on  the  nonnegative  integers, 
let  Ag(/)  -  g(i)  -  g(i  ~  1),  /  >  1,  and  -  g(i)  -  2g(i  -  1)  g(i  -  2),  1  ^  2.  Thus, 

g  is  nonincreasing  (nondecreasing)  iff  Ag  (1)  <  0  O  0)  and  convex  (concave)  iff  A^g(r)  >  0 

«  0). 

Let  /  be  a  real-valued  function  defined  on  C  x  D  C  R".  /is  supermodular  (submodular) 
on  D  if  for  d],  d2  €  D,  /(c,  d\  -  <#2)  /{</.  -  ^2)  >  «)/(c,  d\)  +  fid,  df),  where  - 

denotes  component-wise  minimum  and  -  denotes  component-wise  maximum.  /  has  antitone 
(isotone)  differences  on  C  x  Dif  for  Ci  C2  and  d\  <  d2. 
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/(C,.  (/j)  -/(C,.  </,)  >  (O/iCi.  di)-/(C2.  d,). 

Let  >4'-  >1  -  {fl  €  /llflj-  11. 

Define  A  -  sup  {2Xi(/,  a\)  +  \jU,  a^)). 


A 

4  i 


We  make  the  following  assumptions  for  Model  1 ; 

(Al)  A  <  +<» 

(A2)  4  is  a  compact  sublattice  of  (under  the  usual  partial  ordering  and  relative  topol¬ 
ogy). 

(A3)  r  is  nonincreasing  convex  in  i,  nonincreasing  in  (<i|,  02,  0)  €  4  and  has  antitone 
differences  on  So  x  /4 Also,  r  >  0. 

(A4)  X,  and  K,  are  each  nondecreasing  concave  in  i  and  nonincreasing  on  4'.  X)  has  iso¬ 
tone  differences  in  (<>  n|)  and  X/  isotone  differences  in  (/,  02)- 

(AS)  X|,  X/,  and  rare  each  continuous  on  A'. 

REMARKS:  Given  state  i  and  action  oj,  the  probability  that  the  machine  will  fail  in  a 
small  time  df  is  X/(/,  02) di.  In  this  sense  we  think  of  X/^  as  a  failure  rate  function.  Similarly, 
Xi  is  a  measure  of  the  tendency  of  the  machine  to  "wear  out."  The  fact  that  the  rates  Xi  and  X/ 
are  nondecreasing  in  /  will  play  a  major  role  in  establishing  a  control  limit  rule  for  replacement 
(compare  to  Derman’s  IFR  conditions  (81).  The  monotonicity  of  Xi  and  Xy  in  (oi,  02,  0) 
makes  precise  the  notion  of  viewing  the  action  as  "preventive  maintenance." 

The  concavity  assumptions  on  X)  and  Xy  can  be  given  the  interpretation  that  when  the 
machine  is  new  there  is  a  strong  tendency  for  wear-out  and  failure  to  increase  sharply.  As  the 
machine  breaks  in,  this  rate  of  increase  declines.  This  concavity  assumption  is  used  to  prove 
the  optimality  of  decreasing  preventive  maintenance. 

From  (A3),  letting  /)  >  a  we  have  r(;  -F  1,  a)  —  r(i  +  1,6)  >  r(i,  a)  —  r(i,  b)  which 
says  that  increasing  the  level  of  maintenance  from  o  to  6  incurs  a  greater  cost  increase  in  state 
/■  -F  1  than  in  state  i.  Not  only  does  the  cost  increase  but  the  benefit  decreases  since  from  (A4) 
Xi(t  -F  1,  a)  -  X|(/  -F  1,  6)  <  Xi(i,  a)  -  Xi(/.  b)  (and  similarly  for  Xy). 

THEOREM  1:  There  exists  an  optimal  stationary  policy  for  Model  1. 

PR(X)F:  Note  that  from  (A2),  (A3),  and  (AS)  the  reward  function  is  bounded.  It  is 
then  trivial  to  verify  that  the  conditions  specified  in  [7]  are  satisfied.  (The  functional  operator 
Hg  defined  in  that  paper  turns  out  to  be  a  two  stage  contraction  due  to  the  possibility  of  instan¬ 
taneous  transitions  /  — '  0.  For  unbounded  return  functions  see  Lippman  (211.) 

Functional  Equations 

Using  the  method  described  in  Lippman  (191  (see  also  Kakumanu  [14],  Serfozo  [28], 
Winston  [321,  Howard  [13],  Anderson  (21)  we  can  transform  the  continuous  time  process  to  an 
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equivalent  discrete  time  Markov  decision  process  (DTMDP).  Denote  the  optimal  return  func 
tion  for  the  n-period  horizon  in  the  DTMDP  by  y„,  »  ^ 

Then 

(1)  W  =  0 

and  for  n  >  1 , 

I  Max  |r„(c  0).  r„(/)) 

(2)  ”  I  0)  /-/■ 


For  X  €  {C  Fj. 

To(x.  0)  S  0 

and  for  n  >  1 , 

(3)  T  (x,  0)  —  Max  {/'(O.  a)  —  xia  4-  X|(0,  fl|)  4-  X/(0,  a?)) 

4-X,(0.  a,)F„_,(l)  4-X/^(0,  Oj)  T„-,(F,  0) 

4-  (A  -  X,(0,  a,)  -  X/0.  aj))  K,_,(0))/(A  4-  a) 

Tod)  =  0 

TAi)  -  Max  Jii.  a,  F,_,)/(A  4-  a) 

"  aiA' 

(4)  JU  a,  K,_,)  -  rO,  a)  +  X,(/.  a,)  +  D  + 

K,U  ai)  T„^SF,  0)  + 

(A  -  Xi(i,  a,)  -  X/(i.  aj)) 


(Pnr  „  -  +00  we  write  F  ^  y.  T).  Equations  (2)  assume  that  a  failed  machine  must 
be  replaced  The  presenc^ of  the  lump  sum  rewards  -C  and  -F  requires  special  consideration 

in  th?  discrete  time  recursions  resulting  in  their  somewhat  ^ 

the  functions  T„.  For  a  discussion  on  the  treatment  of  lump  sum  reward  sw  Serfrao  12W  o 
Anderson  12).  In  (2),  T„(C,  0)  is  the  return  if  the  replacement  action  is  chosen,  T„{F.  0)  is 
the  return  if  an  in  service  failure  occurs,  and  r„(i)  is  the  return  otherwise. 

We  denote  the  opiimal  action  in  state  i  by  a*(i.  n)  and  resolve  ties  in  (3)  by  defining 
uMxn) -In?  la  Ta  VU  a.  -  Miut  JU  a,  K,.,))  which  is  well-defined  since  7  is  con¬ 
tinuous  on  A '  and  A  is  compact. 


Finally,  if  a  tie  occurs  in  (2)  we  define  F„(i)  - 

LEMMA  1:  For  0  <  a  <  4-oo  and  i  €  So,  T„iF.  0)  <  K«(i). 

PROOF;  Equation  (2)  implies  F,(i)  >  r„(C  0)  >  r,(f.  0)  since  F  >  C  P 


LEMMA  2:  For  0  <  a  <  4-oo,  F,  is  nonincreasinf  in  /  €  Sq. 
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PROOF:  We  prove  the  lemma  for  finite  n  (by  induction).  The  infinite  horizon  case  fol¬ 
lows  by  taking  limits  (F„  —  F).  For  n  —  0  the  result  is  trivial.  For  n  >  0  it  suffices  to  show 
that  A7„(/)  <  0.  Define  a  -  a*{i)  «•  (ai.aj.  0).  From  (3),  (4),  and  the  definition  of  a, 

(5)  (A  +  a)  [T„ii)  -  T„{i  -  1)1  <  JU  a.  F„_,)  -  Jii  -  1.  a.  F„_,) 

<  AF„_,(,  +  1)X,(/.  a,) 

-F  AF„_,(/)  (A  -  k]ii  -  a,  fli)  -  kfii  -  1,  02)] 

(6)  <  0. 

Inequality  (5)  follows  from  \/(/.  02)  >  K/ii  —  1,  a^)  and  T„^\(F,  0)  -  F„_|(;)  <  0  (Lemma 
1),  Inequality  (6)  follows  from  the  definition  of  A  and  the  inductive  hypothesis.  □ 

THEOREM  2  (Control  limit  rule  for  replacement):  There  exists  /*,  0  <  /•  <  such 
that  for  /■  >  i*,  ajf/)  —  1  (i.e.,  the  optimal  action  is  replacement). 

PROOF:  From  (2)  it  suffices  to  show  that  T  is  nonincreasing  on  So.  In  the  proof  of 
Lemma  2  we  showed  that  A7i(/)  ^  0  for  finite  n.  Using  AF(/)  ^  0  (Lemma  2),  the  proof 
that  £kT„ii)  <  0  holds  in  the  same  way  for  n  —  +00.  Now  set  i*  —  inf{i:  T(C,  0)  >  r(/)). 


In  order  to  establish  the  optimality  of  nonincreasing  preventive  maintenance  we  first  need  the 
following  lemma. 

LEMMA  3:  For  0  <  ff  <  +«>,  F„  is  convex  in  /. 

PROOF:  The  convexity  of  V  will  follow  from  that  of  V„(n  <  -(-«>)  by  taking  limits  on  n. 
For  n  <  +00  we  employ  induction.  Since  the  maximum  of  convex  functions  is  convex,  from 
(2)  it  suffices  to  show  that  T„  is  convex.  Since  Kq  =  0  the  convexity  of  Fq  is  trivial.  Assume 
then  that  F„_|  is  convex.  To  show  that  T„  (and,  therefore,  F„)  is  convex  we  need  A^r„(i)  - 
-  T„U  -  D)  -  (r„(/  -  1)  -  T„(i  -  2))  >  0,  i  >  2.  Define  a  -  o*(i  -  1, 

n)  -  (fli,  02- 0)-  Ffon'  ^3)  we  have 

(7)  (A  -F  a)  lr„(/)  -  T„{i  -  1)1  >  JU.  a.  F^_,)  -  JU  -  1,  o.  F„_,) 

(8)  (A  +  a)  IT„U  -  1)  -  T„U  -  2)1  <  JU  -  1,  0,  F„_,)  -  /(/  -  2,  a,  F„_,). 

From  (7)  and  (8)  in  order  to  establish  A^r„(/)  >  0  it  suffices  to  prove  that  AV(i,  a)  = 
JU.  a.  F„_,)  -  2JU  -  1,  a,  F„_,)  -F  JU  ~  2.  a.  F„_,)  ^  0. 


From  (4), 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 


AV(/,  a)  -  rU.a)  -  2rU  -  1,  a)  +  rU  -  2,  a) 

-F  X\ii,  a{f  F„_|(/  -F  1) 

-FAF„_,(/)[A-2X,(i-  1.  a,)l 
-FAF„_|(i  -l)(-A-F\,(/-2,  ai)l 
■F  0)  (Ay(/,  —  2kfii  ~  1,  a2)  +  k/ii  ~  2.  a2)l 

-F  F„_i(/)  [— X/(/,  a2)l 
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(15)  +  1)  (2X/i- 1.  02)1 

(16)  +  y„~iU  -  2)  I-X/(/  -  2,  02)1 

Consider  the  last  four  terms  of  the  above  sum  ((13)-(]6)).  From  Lemma  1,  T„^\(F,  0)  ^ 
-  1)  and  using  the  concavity  of  X/  we  see  that  the  sum  of  (13)-(16)  is  greater  than  or 
equal  to 

(17)  AK,_,(;)(-Xy(/.  02)1  +  F„_,(/  -  1)  (X/(/  -  2,  02)!. 

Replacing  (13)-(16)  by  (17),  using  the  convexity  of  r,  the  definition  of  A  and  the  induction 
hypothesis  that  V„_^  is  convex  (AK„_,(/)  >  AF„_,(/  -  1)),  we  see  that 

AV(/,  0)  >  AF„_,  (/-  1)  [X,(/.  0,)  -  2X,(/  -  1,  o,)  +X,(/ -  2,  o,) 

+  Xy(<  -  2,  O2)  -  kf(i.  02)! 

>  0 

where  the  last  inequality  follows  from  the  concavity  of  X|  and  the  monotonicity  of  X/.  □ 


THEOREM  3:  Prior  to  replacement,  the  optimal  preventive  maintenance  level  is  a  nonin¬ 
creasing  function  of  /,  i.e.,  for  0  <  /  <  /*,  a*{i)  >  a*(i  +  1). 

PROOF.  From  Theorem  6.2  of  [301  and  assumption  (A2)  it  suffices  to  show  that  (1)  for 
each  /,  J  is  supermodular  in  a  €  >4'  and  (2)  J  has  antitone  differences  on  x  Sq.  (1)  follows 
from  the  supermodularity  of  r  and  the  fact  that  J  —  r  is  a  separable  function  of  a\  and  ai- 
Let  a  -  (ai,  a^,  0)  <  6  -  (bj,  62*  0)  ^  Define  A,,,.;,  =  J(i  +  \  ,  b,  V)  +  J(i,  a,  V)  - 
J(i  -(-  I.  a.  n  -  J(i,  b.  V). 

Then, 

(18)  A,.a,(,  -  rU  +  I,  b)  +  r(i,  a)  -  r(i  -t-  1,  a)  -  rii,  b) 

(19)  -t-AK(/  +  2)lXi(i  +  l.  6,)-X,(/  +  l,  a,)l 

(20)  -1-AF(/ -E  l)lX|(i,  a) -X,(i,  6)1 

(21)  +  (r(F.  0)  -  Vii  +  D)  (X,(i  +  1,  62)  -  kfd  +  1.  a2)l 

(22)  +  (nr,  0)  -  VU))  lXy(i.  aj)  -  kfU.  62)1 

(23a)  <  AF(/  +  l)lx,(i  +  1.  6,)  -  X,((  +  1,  a,)  +X|(/.  a,)  -  X,(f,  6)1 

(23b)  +  (TiF.  0)  -  Vii  +  D)  Ikfd  +  1,  62)  -  k  f(i  +  1.  a2) 

kfii,  02)  -  k 62)1 

(24)  <  0. 

The  inequality  yielding  (23a)  and  (23b)  results  from  the  following:  From  (A3),  summand  (18) 
is  nonpositive;  use  the  convexity  of  V  and  the  fact  that  X 1  is  nonincreasing  on  -4 '  to  obtain 
(23a)  from  (19)  and  (20).  Now  use  Lemma  2  and  the  fact  that  k,  is  nonincreasing  on  >4'  to 
obtain  (23b)  from  (21)  and  (22). 


Inequality  (24)  follows  using  Lemma  1,  and  assumption  (A4)  that  Xj  and  X2  have  isotone 
differences. 
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This  completes  the  specification  of  the  structure  of  the  optimal  maintenance/replacement 
policy  for  Model  1.  Note  that  the  results  hold  for  the  special  cases  =  0,  or  X ,  =  0.  Also 
observe  that  from  (4),  JU,  a,  V)—  r(i,  a)  +  Xi(/,  aj)Ay(i  +  1)  +  Xy(/,  02)  (T(F,  0) - 
V(i))  +  A  VU).  Thus,  if  r  is  convex  in  <2,  and  X 1  and  X  f  are  concave  in  a,  J  will  be  convex  in 
a.  (Recall  Lemmas  1  and  2.)  In  this  case,  a*  is  a  bang-bang  policy  for  /  €  Sq. 

a*  —  sup  (a  €  i4')  or  a*  —  inf  [a  €  A'\.  (A'  is  compact  by  assumption  (A2).)  Xj  concave 

nonincreasing  in  a\  means  that  small  increments  in  above  inf  A '  have  little  effect  in  decreas¬ 
ing  X|  (and  similarly  for  02  and  X y).  However,  r  convex  nonincreasing  in  a  implies  that  the 
increase  in  cost  for  such  an  increment  is  great.  Whether  fl*(0)  —  supy4'  or  fl*(0)  —  inf  .4'  will 
depend  on  how  effective  preventive  maintenance  is  against  protecting  the  new  machine  relative 
to  the  cost  of  the  machine. 

MACHINE  MAINTENANCE  -  MODEL  2 

The  specification  of  Model  2  is  identical  to  Model  1  regarding  the  state  space,  reward 

structure,  and  optimality  criterion.  However,  Model  2  differs  in  the  nature  of  machine 

deterioration.  Preventive  maintenance  is  represented  by  a  pair  a  —  (oi.  02, 0)  €  A  Q  R^. 
The  transition  /  —  /  (failure)  occurs  at  rate  X ,(/.  02)-  Also,  the  machine  may  deteriorate  to 
state  /■  -F  jfrom  state  /atrateXO.  o\),J  €  {1,2.  ...). 

Suppose  the  previous  assumptions  in  Model  1  on  /I,  r  and  X y  hold  here.  In  addition,  we 
assume  that  \(J,  •)  is  nonincreasing  and  continuous  on  A'  for  j  €  Sq.  Also,  assume  that  A  = 
sup  {IX (y,  a|)  -F  X/(/,  ay))  <  +<».  Then  routine  application  of  the  techniques  used  for  Model 
I  establishes  that  the  optimal  maintenance/replacement  policy  for  Model  2  exhibits  the  same 
monotonic  structure.  Since  the  proofs  are  completely  straightforward  we  omit  the  details. 

MACHINE  MAINTENANCE-MODEL  3 

Again  Model  3  differs  from  Model  1  only  in  the  nature  of  the  deterioration  process. 
Specifically,  denote  an  action  by  a  =  (aj.  02,  oj)  €  A  C  R^.  ay  =  1  indicates  replacement  as 
before.  Suppose  ay  =  0.  Then  failure  i  —  /occurs  at  rate  Xy(/.  02)  as  before.  In  addition  the 
machine  may  also  deteriorate  from  state  /to  a  state  /  -F  mii,  aj,  p)  at  rate  KdFip).  Here  pis  a 
random  variable  taking  values  ifi  c  set  P  and  having  distribution  t.  Let  Z  denote  the  set  of 
integers.  Then  at  is  a  map  m:  Sq  x  A  x  P  ^  Z.  We  think  of  p  €  /*  as  a  measure  of  the  mag¬ 
nitude  of  a  randomly  occuring  shock.  The  time  between  shocks  is  exponential  with  mean  1/X. 
The  three  variables  /.  ai,  and  p  together  determine  the  subsequent  "damage"  mii.  a^p)  to  the 
machine. 

We  make  the  following  assumptions: 


(Bl)  A  =  sup  (X/(/,  02)}  +  X  <  -Foo 

(B2),  (B3)  same  as  (A2)  and  (A3),  respectively. 

(B4)  The  conditions  on  X  are  the  same  as  stated  in  (A4). 

(B5)  The  function  m  is  nonincreasing  in  aj  ('preventive  maintenance"),  is  nondecreasing 
in  I  €  So,  and  has  isotone  differences  on  Sq  x  A. 
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(B6)  m,  X/,  and  r  are  each  continuous  on  A. 

These  assumptions  are  to  be  interpreted  in  a  manner  similar  to  the  discusssion  following 
(A1)-(A5)  of  Model  1. 

It  is  easy  to  establish  Theorem  1  for  Model  3.  We  now  present  the  recursions  for  the 
present  model. 

Recursions  for  Model  3 


The  DTMDP  recursions  equivalent  to  the  CTMDP  are  the  following: 
(25)  Ko(i)  =  0 

and  for  1  <  w  <  +<», 

Max(r„(C.O).  r„(/))  /  / 

T„(f.  0)  /  -/ 


(26)  y„(i) 

For  X  €  (C  Fl, 


Toix.  0)  =  0 


and  for  w  >  1,  with  /I'  -  >4  -  {a  €  ^4 103  »•  0), 

r„(x,  0)  -  Max  {r(0,  a)  -  x(a  +  X  +  X/(0.  oj)) 

a^A' 

+  X  F„_,(m(0,  Oi,  p)dF{p)  +X/(0,  a2)T„-\{F,  0) 
+  (A  -  X  -  X/(0,  fl}))  K„_)(0))/(A  +  a) 


for  n  >  I, 

(27)  T„U)  -  Max  J(i.  a.  F„_,)/(A  +  a) 

a^A 

(28)  Jii,  a.  F„_|)  -  r(l,  a)  +  \  (/  +  m{i,  a^,  p))  dFip) 

+  Xf(i.  aj)  T„.f{F.  0) 

+  (A  —  X  -  Xf(i.  ai))  F„_|(/) 

As  before  for  n  —  +<»  we  write  —  F  and  T„  —  T,  and  a*{i,  n)  denotes  the  optimal 
maintenance  action  In  state  i. 

LEMMA  4:  For  0  <  «  <  +<»  and  /  €  So, 

TJF.  0)  <  V„{i). 

PROOF:  The  proof  is  similar  to  the  proof  of  Lemma  1.  □ 


LEMMA  5:  For  0  <  «  <  +»>,  is  nonincreasing  in  /  €  Sq. 
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PR(X)F;  As  before,  the  infinite  horizon  case  will  follow  from  the  finite  horizon  result. 
The  result  is  trivial  for  «  0  since  Vq  =  0.  Assume  A  <  0  for  i  €  5o.  From  (26)  it 

suffices  to  show  Ar„(/)  <  0.  Let  a  »  a*{i,  n)  -  (a\,  oj,  0)  6  y4'. 

From  (27), 

(29)  \T„{i)  <  J(i,  a.  K„_,)  -  J(i  -1,0,  F„_,) 

“  A/-(/,  o) 

+  X  F„_|  (/  +  mil.  fl,,  p))  -  K„_,(/  -  1  +  m(i  -  1,  Oi,  p))  dF{p) 
+  X/(/,  oj)  [r„_,(f.  0)  -  K„_,(/  -  1)1 

-  x,((  -  1.  flj)  {r„_,(f.  0)  -  F„_,(/  -  1)1 

+  (A  -  X)AF„.,(/) 

<  Ar(j,  fl)  + 

\p  J  [K„_,(;  +  m(/,  fl],  p))  -  K„_,(;-  1  +  m(i  -  1,  fli,  p))l</F(p) 
+  (A  -  X)AF„_,(/) 

(30)  <  0. 

Inequality  (29)  follows  from  Lemma  4  and  the  monotonicity  of  Xf  in  /.  To  obtain  the  last  ine¬ 
quality,  (30),  use  (A3)  on  r,  (B5)  on  M,  and  the  induction  hypothesis  that  is  nonincreas¬ 
ing,  along  with  the  definition  of  A.  g 

THEOREM  4  (Control  limit  rule  for  replacement):  There  exists  i*,  0  <  /*  <  -l-«>  such 
that  for  /  >  /•,fl*(i)  =  1  (replacement). 

PROOF;  The  proof  is  similar  to  the  proof  of  Theorem  2.  □ 

LEMMA  6:  For  0  <  w  <  +°o,  y„  is  convex  in  i. 

PROOF:  As  in  Lemma  3,  it  suffices  to  show  that  T„  is  convex.  Kq  =  0  so  assume  that 


is  convex. 

Let  fl  —  fl*(/  -  1,  n)  —  (fl|,  fl2.  0)  and 

^^JU.  fl)  s  JU  a.  K„_,)  -  2JU  -  I.  fl.  K„_,)  +  Jii  -  2.  fl.  F„_,). 

From  (27), 

>  AV(/,  fl) 

(31) 

-  AV(/,  fl) 

+  X  lF„_,(/  +  mil.  fl,,  p))  -  2K„_,(i  -  1  -F  m(/  -  1,  fl,,  p)) 

(32) 

-F  y„-\ii  -  2  +  m(i  -  2.  fl,,  p))l  dF(p) 

(33) 

-F  A2X/(/,  flz)  r„_,(F,  0) 

(34) 

-F  K„_,(/)lA  -  X  -  X/(/,  flj)! 

(35) 

-F  y„_,(i  -  1)  1-2A  +  2X  +  2Kf(i  -  1.  aj)! 
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(36)  +  -  2)  [A  -  X  -  X^(/  -  2.  flj)]. 

From  (B3),  A^r(/,  a)  >  0.  Define  mo=  /  +  m(i,  fli,  p),  mi  -  /  -  1  +  m(/  -  1,  a;,  />),  and 
m:  “  /  -  2  +  m(/  -  2,  ai,  p).  From  (B5)  mo  >  mi  >  m2  and  m  concave  in  /  ^  mo  - 
mi  ^  mi  -  m2.  Since  F„_|  is  convex  (the  inductive  hypothesis)  and  nonincreasing  (Lemma 
5),  K„_,(mo)  -  K„_i(mi)  >  F„_|(mi)  -  F„_|(m2)  which  establishes  that  the  integrand  in  (32) 
is  nonnegative. 

Finally,  the  proof  that  the  sum  of  the  last  four  terms  (33)-(36)  is  nonnegative  is  the  same 
as  in  Lemma  3.  Thus,  A^J(/,  a)  >  0. 

THEOREM  5:  Prior  to  replacement  the  optimal  preventive  maintenance  level  is  a  nonin¬ 
creasing  function  of  i. 

PROOF:  As  in  the  proof  of  Theorem  3  it  suffices  to  show  that  (1)  for  each  i,  J  is  super- 
modular  in  a  €  A'  and  (2)  J  has  antitone  differences  on  A'  x  Sg.  (1)  follows  as  before.  To 
establish  antitone  differences  we  first  invoke  (B3)  on  r.  Also,  the  proof  that  J  has  antitone 
differences  in  i  and  a2  is  the  same  as  in  Theorem  3.  It  thus  suffices  to  prove  that  the  integrand 
in  (28)  exhibits  antitone  differences  in  i  and  oi.  Let  oi  <  bi.  Define 

mi  =  /  -i-  1  -!■  m (/  -t-  1 ,  b\,  p),  m2  =  »  +  1  +  m(/  -f  1,  Oi,  p),  m^  =  i  +  m(i,  b\,  p),  and 
=  i  +  m(i,  a\,  p).  From  (B5)  mi  <  m2,  mj  <  mi,  m.i  <  m4  ^  m2.  Define 
gii,  ai)  =  y{i  -I-  m(i,  a^,  p)).  Then 

(g((  -I-  1,  61)  -  g(i  +  1.  <ri)l  -  (g(/.  f>i)  -  gii.  0|)] 

=  [F(mi)  -  F(m2)I  -  (F(m,)  -  Ffmjl  <  0. 

The  inequality  follows  from  the  fact  that  V  is  nonincreasing  convex  and  the  fact  that 
m2  -  mi  <  m4  -  m3  (namely  (B5)  which  assumes  that  m  has  isotone  differences  in  (/,  a^)). 
This  completes  the  proof.  p. 
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ABSTRACT 

A  new  bivariate  negative  binomial  distribution  is  derived  by  convoluting  an  existing 
bivariate  geometric  distribution;  the  probability  function  has  six  parameters  and  admits  of 
positive  or  negative  correlations  and  linear  or  nonlinear  regressions.  Given  are  the  mo¬ 
ments  to  order  two  and,  for  special  cases,  the  regression  function  and  a  recursive  formula 
for  the  probabilities.  Purely  numerical  procedures  are  utilized  in  obtaining  maximum 
likelihood  estimates  of  the  parameters.  A  data  set  with  a  nonlinear  empirical  regression 
function  and  another  with  negative  sample  correlation  coefticient  are  discussed. 


1.  INTRODUCTION 


In  this  paper  we  develop  a  new  bivariate  negative  binomial  (bnb)  distribution  by  convolv¬ 
ing  a  certain  bivariate  geometric  distribution.  The  univariate  negative  binomial  with  parameters 
V  >  0  and  #  >  0  is  defined  (Johnson  and  Kotz  I6l)  as  the  distribution  of  a  random  variable 
(r.v.)  X  for  which 
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x]~ 


r(x  +p) 

I 

X 

e 

X  +  V  -  1 

1 

V 

0 

x!r(p) 

1  +e 

1  +» 

X 

I  +fl 

1  +0 

0.  1.  2.  ... 


where 


A  +  V  —  1 
X 


teristic  function  is 


shall  be  taken  to  be  defined  as  the  ratio  of  gamma  functions;  the  charac- 


[1  +0(1  -  e"))"". 


The  mean  and  variance  of  X  are  vO  and  u0{\  +0),  respectively.  For  v  1  we  have  the 
geometric  distribution.  We  shall  find  it  more  convenient  to  use  probability  generating  functions 
than  characteristic  functions  because  of  the  discrete  nature  of  the  random  variables  we  consider. 
The  correspondence  between  characteristic  functions  and  probability  generating  functions  is 
effected  by  letting  u  -  e''  which  gives 

'This  research  was  performed  when  the  author  was  affiliated  with  the  United  States  Air  Force  Academy.  Colorado 
Springs.  Colorado. 
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(2)  «<.(«)-  [1 +«(1  -  m))-". 

Another  common  representation  is  to  let®  -  p/(l  -  p),  or  equivalently,  p  —  S/(l  +»), 
in  (1 )  and  so 

(3)  Pr[X~  xl°  (1  -  p)‘'p^  X- 0,1,2.... 

jf  !r  (i/ ) 

(4)  «(m)=  -  1  +  7-^ 

1  -  p  I  -  p 

This  latter  representation  is  referred  to  as  a  negative  binomial  distribution  with  parameters  v 
and  p.  We  use  both  representations  throughout. 

The  probability  function  of  a  bnb  distribution  (Mardia  [9))  (or  negative  multinomial 
(Johnson  and  Kotz,  (61)  is 

(5)  Pr[X  =  X.  K  =  vl  =  ^  ^  (1  -  p  -  qY  p’‘q\  x,  y  =  0.  1.  2.  ... 

x!>'!r  (v ) 

where  v  >  0,  p  =  0/[l  +  (a  +  lW],  q  =  a  p,  0<p<  1,  0<  q  <  1,  and  0  <  p  +  q  <  1. 
The  probability  generating  function  is 

(6)  4>(u.  v)  =  £[u^vn  =  11  +«(1  -  m)  +ae(l  -  v))-". 

Guldberg  [5]  introduced  this  distribution  and  Bates  and  Neyman  [3]  fitted  it  to  several  data 
sets.  We  designate  this  bnb  distribution  as  G  —  B  ~  N(a.  0.  v).  The  distribution  admits  only 
of  positive  correlation  and  linear  regressions. 

Certain  data  sets  do  not  exhibit  empirical  regressions  which  are  linear  nor  do  some  data 
sets  show  positive  correlation  and  so  it  is  natural,  for  these  cases,  to  work  with  a  bivariate  pro¬ 
bability  function  which  allows  for  nonlinear  regressions  or  negative  correlations  or  both.  The 
classical  Bates  and  Neyman  paper  exhibited  empirical  data  which  would  seemingly  be  best  fit  by 
regression  curves  which  were  obviously  nonlinear,  and,  consequently,  their  results  were  not 
entirely  satisfactory.  Furthermore,  in  Table  3  we  provide  some  new  bivariate  data  related  to 
aircraft  flight  aborts  which  has  a  negative  sample  correlation  coefficient.  We  would  thus  be 
reluctant  to  use  a  bivariate  distribution  which  did  not  admit  of  the  possibility  of  both  negative 
correlation  and  nonlinear  regressions  to  describe  the  observed  abort  phenomenon.  These  few 
instances  (and  several  others  we  do  not  provide)  indicate  there  is  a  need  for  a  study  of  the  pro¬ 
perties  and  potentialities  of  new  multivariate  negative  binomial  distributions  which  provide  for 
more  flexibility  than  those  which  have  heretofore  been  discussed  in  the  literature.  We  also  iso¬ 
late  some  situations  which  call  for  even  more  general  distributions  than  we  consider  herein. 
Before  we  perform  the  data  analysis  we  discuss  an  inventory  model  which  generates  the  new 
distributions  and  examine  the  resulting  distributions  in  some  detail. 

2.  A  STORAGE  MODEL 

Consider  the  storage  system  consisting  of  two  facilities,  one  of  which  is  used  to  store 
commodity  A  alone,  the  other  to  store  commodity  B  alone.  At  each  epoch  of  time, 

n  -  1.  2 . facility  1  receives  a  random  amount  of  commodity  A,  say  a„,  and  facility  B 

receives  a  random  amount  of  commodity  B,  say  b„.  Take  the  a„  and  b„  to  be  mutually 
independent,  identically  distributed  random  variables.  Let  the  total  of  A  and  B  at  epoch  n  be 
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A„  and  B„.  The  total  A„  and  B„  are  related  to  the  totals  A„-i  and  B„-\  in  the  following 
fashion.  Between  epoch  n  —  1  and  n  the  totals  /4„_i  and  B„_|  may  randomly  be  reduced  to 
zero  or  remain  intact.  The  manner  in  which  A„-i  and  B„-\  are  carried  over  to  the  nth  epoch 
determines  storage  characteristics.  The  system  equations  are 

a,  +  A„.i 
“  bn+C„  B„-x. 

and  without  loss  of  generality  we  take  Aq^  So  ““  0-  There  are  four  cases  of  interest; 

(1)  Pr(|,-{„- 0)=  a 

Pr  -  1-fl. 

(H)  Pr  0.  0)=  n 

Pr  1.  0)=  6 

Pr(«,=  l.i„=  i)=  =  1-a  -  6. 

(III)  Pr  0.  C,=  0)-  a 

Pr  0.  1)=  c 

Pr(^„-  l.i,  -  D-  rf  -  1-  a  -  c. 

(IV)  Pr  (I,- 0.  0)-  a 

Pr  (^,=  1.  0)-  6 

Pr  (|„-  0.  D-  c 

Pr  (^„-  1.  C„“  !)“</-  1  -  a  -  6  -  c. 

For  example,  (I)  implies  that  the  contents  of  facilities  A  and  B  are  either  both  discharged  or 
both  kept  in  the  time  between  epoch  n  —  1  and  n.  (11)  implies  that  the  contents  of  both  are 
simultaneously  discharged  with  probability  a;  the  contents  of  B  is  discharged  while  those  of  A 
are  kept  with  probability  b\  and  the  contents  of  both  A  and  B  are  kept  with  probability  d. 
Finally,  take  both  a„  and  b„  to  be  geometric.  Provided  b  +  d  <  \,  c  +  d  <  1,  the  system 
defined  by  the  above  system  equations  soon  reaches  steady  state  and  is  governed  by  a  bivariate 
geometric  probability  distribution  (Paulson  and  Uppuluri,  (III).  If  there  are  m  identical  sys¬ 
tems  of  this  type  and  the  quantities  A  and  B  are  summed  across  the  m  systems,  a  bivariate 
negative  binomial  distribution  results.  Curiously,  cases  (I),  (II),  and  (111)  give  rise  to  infinitely 
divisible  bivariate  distributions  (as  we  show  later)  whereas  (IV)  does  not  in  general  The 
analytical  details,  starting  from  the  steady  state  system  whose  behavior  is  defined  by  a  func¬ 
tional  equation  in  the  probability  generating  function,  are  given  in  the  next  section. 

3.  THE  NEW  BIVARIATE  NEGATIVE  BINOMIAL  DISTRIBUTION  AND  PROPER¬ 
TIES 


Paulson  and  Uppuluri  (111  showed  that  the  bivariate  r.v.  (X,  K)  (identify  X  and  Y  with 
the  steady  state  A„  and  B„,  respectively),  where  each  element  in  the  pair  is  defined  on  the  non¬ 
negative  integers,  has  a  bivariate  geometric  distribution  if  its  probability  generating  function, 
0  (u.  v),  satisfies  the  functional  equation 
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it  can  be  shown  that  Theorem  12.2.3  of  Lukacs  [8]  can  be  appropriately  extended  to  yield 
the  result  that^iu,  v)  is  infinitely  divisible,  or  equivalently,  ^|(a.  is  infinitely  divisible  (see 
also  Block,  Paulson,  and  Kohberger  [4]. 

Thus,  the  v-fold  convolution  of  /f|(x  y)  with  itself  yields  a  bnb  distribution  which  we 
designate  g,{x.  y)  where  v  need  not  be  restricted  to  integer  values.  The  probability  generating 
function  (u,  v)  of  g„(x  y)  is 

(14)  ^uiu,  v)  -  ((1  -  d'hit  i<l<2)‘'(l  -  dijt  ill's)'" 

-  ((1  -  dkl*  ill's)" (1  +p{hliflt2+  ^ ^  ^ ^  (tAl*  iil»s)^  +  . . . ) 
which  gives  in  the  same  manner  as  above 

(15)  g^(x.  y)  •  (1  -  d)" h](x)fi2(y)  Fix  +  v,  y  +  i';v;z), 
where 

|»<  +  X  -  1 

”1  X  (!“/>)"  P* 

IV  +  y  —  1 

^  (1  -  qY  q" 

z  -  </(l  -  p)  (1  -  q) 

and  X.  y  -  0.  1.  2 .  It  is  clear  by  construction  that  the  marginals  of  g^ix,  y),  say  gxi'('>f) 

and  g}yiy),  are  negative  binomial  with  0|  =  pliail  -  p))  and  Os"  Q/iail  -  q))  as  may  be 
determined  from  (8)  with  b  -  c  *  0. 

The  infinite  divisibility  of  ;„(x.  y)  is  a  result  of  considerable  appeal  in  applications  since  v 
need  not  be  restricted  to  integer  values. 

We  remark  in  passing  that  the  distribution  (15)  is  the  discrete  analogue  to  the  Wicksell- 
Kibble  bivariate  gamma  distribution.  The  k-variate  extension  may  be  readily  derived  as  above 
and  has  probability  distribution  (in  obvious  notation) 

k 

g^ix\.  Xi . •**)  “  Cik)  Fn_i(xi  +  v . x*+v;v .  I';  d  11(1 -A)). 

1-1 

with 

*  (v  +x;-  ll 

C(k)-n  V  (1- P')''(Py)  '• 
j-\  I 

Next  we  consider  the  case  with  b  and  c  different  from  zero  in  (7).  For  notational  con¬ 
venience,  set  (^i(u,  1)  ■■  «Ai  and  ^2(1.  v)  —  ^2.  Denote  the  probability  function  corresponding 
to  (7)  by  /](x,  y).  Observe  that  when  6  -  c  -  0,  /i(x.  y)  -  gi(x.  y).  The  (integral)  m-fold 
convolution  of  /i(x,  y)  with  itself  has  probability  generating  function  0„(u,  v),  probability 
function  /„(x,  y),  and  marginal  probability  functions  fxm^x)  and  /r„(y),  respectively.  With 
some  minor  calculation  we  find 
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(16) 


v) 


(1  -  d)>ltfll2 


1  -  (Nifhi  I  \\  -  d 
From  (8)  or  (16)  it  is  easily  shown  that 
(17a)  <6„(m.  1)  -  (1  +«,(!- m))  *. 

(17b)  <6m(l.  v)-  (1 +fl2(l  -  v))-*. 

and  accordingly. 


- d>  I  4"  - df  j 

\  -  d  '  \  -  d^^ 


(18a) 


(18b) 


fxm^x) 


r  (jf  4-  /» ) 

r(x  4-  i)r(m) 


f  r  )  =  r  (.w  4-  rw ) 
rcv  4- i)r(/M) 


for  jf  =  0.  1 . .V  =  0,  1 . and^i  and  #2  as  given  in  (8c)  and  m=  1,  2.  ...  Define 

\  1  =  a/(l  —  rf).  X 2  =  6/(1  —  d),  Xj»  c/(l  —  d)  in  (16).  Since  products  of  probability  gen¬ 
erating  functions  correspond  to  convolutions  of  probability  functions,  an  application  of  the  tri¬ 
nomial  expansion  to  (16)  coupled  with  a  termwise  inversion  of  products  of  the  form  (17)  with 
(15)  gives 


THEOREM  1:  The  probability  function  of  (16)  with  a.  6.  c.  rf  >  0,  a  4-  6  4-  c  4-  rf  = 
b  +  d  <  I,  c  +  d  <  l,is 


(19) 


/„{x.y)~‘  X  y)  *./■>, Cv). 

a.0  y  “  4'  -r  • 


1, 


-  I 

n.B.y 


m! 

a  !> ! 


t  t  fxp(x  -  O  Ayiy  -  r,)  g„{^ .  1,). 


where  T  runs  over  all  a ,  /3 .  -y  >  0  such  that  a  4- 13  4-  y  =  m.  The  operator  *  for  convolution 
over  X  is  defined  for  two  functions  6i(x,  y)  and  62(x.  y)  by 


6 1  *  62  -  X  ~  ^  -  3')- 

«-o 

y 

The  operator  *  is  defined  similarly.  When  /3  =  0  or  y  =  0  in  (19),  fxo^x)  and  /joly)  are  taken 
to  be  unity. 


The  distribution  f„(x,  y)  will  be  henceforth  referred  to  as  the  BNB(a.  6.  c.  p,  q,  m)  dis¬ 
tribution.  The  distribution  g„(x.  y),  only  when »'  =»  m  an  integer,  is  a  special  case  of  f„{x,  y); 
the  distribution  f„(x.  y)  is  not  in  general  infinitely  divisible.  The  BNB(a.  b.  c,  p,  q,  m)  dis¬ 
tribution  is  the  discrete  analogue  to  Paulson's  (10]  bivariate  gamma  distribution. 


As  a  direct  consequence  of  Theorem  I  we  obtain  a  closed  form  representation  of  Paulson 
and  Uppuluri's  [11]  bivariate  geometric  distribution  as 


COROLLARY  1;  For  m  -  1,  <  I,  the  BNB(a,  b,  c.  p.  q.  \)  distribution  is 
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(20) 


/i(x  y) 


^  «l(x  J')  +  * 


1  -  d 


1  -  d 


1  -  d 


1 


1+0, 

1 


0 


1  +fl 


1  +0, 


1  +0- 


'  g\(x.  >■) 


•«i(x,  y). 


The  requirement  that  the  shape  parameter  m  of  /„(x,  >’)  in  (19)  be  integer  valued  limits 
the  practical  utility  of  the  probability  function.  We  are  thus  led  to  seek  a  representation  for 
which  the  shape  parameter  may  take  on  arbitrary  real  values.  This  is  equivalent  to  first  deter¬ 
mining  whether  y)  is  infinitely  divisible,  and  if  so,  to  secondly  determining  an  expression 
which  is  not  tied  to  integer  values  m.  In  general,  the  BNBia,  b,  c.  p,  q,  m)  distribution  is  not 
infinitely  divisible  (Block,  Paulson,  Kohberger  [4])  and  no  such  representation  will  exist.  The 
BNBia,  b.  O.p,  q,  v)  and  BNBia,  0.  c.  p.  q.  v)  distributions  are,  however,  infinitely  divisible 
as  are  the  BNBia,  0.  0.  p.  q.  i>)  distributions. 

THEOREM  2.  The  BNBia.  b.  0.  p,  q,  v)  and  BNBia.  0.  c.  p,  q,  v)  distributions, 
b  +  d  <  \.  c  +  d  <  d  <  l,i'>0,  are  infinitely  divisible. 


PROOF:  In  (16)  replace  m  by  i'  and  write  the  resulting  probability  generating  function 
when  c  =  0  as  (c  =  0).  Then  it  follows  that 


(21) 


<t>Jc=  0) 


(1  -  dhlffli2 

tf 

a 

1  -  (bin 

1  -  dtlifli  2 

1  -  d 

1  -  ib  +  d)<in 

The  term  in  square  brackets  is  infinitely  divisible.  The  term  in  braces  is  infinitely  divisible  if  it 
is  a  probability  generating  function  for  arbitrary  >  0.  But 


(22) 


(1  -  d«|<,) 


1  -  (6  -t-  d)«l»  I 


exp 


•'jloglj^^ 


1  - 


ib  +  d)>l>; 

explv  (logd  -  cblf])  -  logd  -  ib  +  dM  i))l 


exp! 


i^Tiib^  dV-  d')  — 
J 


<30  n 

t/ 

n  -  1 

>+  I  I 

«-]  A-1 

k 

n  —  k 

bHb  +  d)""*i|.r 


•  \  ^  hib.  d,  n. 

n*  \ 


say,  on  successive  expansion  of  logd  —  dibi),  logd  —  ib  +  dH])  and  the  exponential  func¬ 
tion.  For  all  1/  >  0,  0  <  6  <  1,  0  <  6  -E  d  <  1,  bib.  d.  n,  v)  >  0.  An  application  of 
Theorem  12.2.3  of  Lukacs  [8I  yields  the  infinite  divisibility  of  the  probability  generating  func¬ 
tion  (ad  -  <W»i)/((l  -  d)  (1  -  (6  +  d)<i(,)))'.  Since  4>^ic~  0)  is  a  product  of  infinitely 
divisible  probability  generating  functions,  it,  too,  is  infinitely  divisible. 
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The  constructive  nature  of  Theorem  2  provides  the  representation  we  seek.  The  term  [•] 
in  (21)  has  probability  function  gjx.  y).  The  term  I  )"  in  (21)  has  probability  function  given 
by  the  termwise  inverse  of 

a 


\  +  ^  hib.  d,  n,  vltli ") 
\ 


1  -  d 

Hence,  the  inverse  of<^„(c  —  0)  of  (21)  is 


(23) 


/„(x  j'lc  -  0)  - 


1  -  d\ 


i  «,.(x  y) 


+  sAx.  y)  *  X  ~ 

«—  I 


r^r'l 


We  thus  have  proven 

COROLLARY  2;  The  BNB(a.  b.  0.  p.  q,  v)  distribution,  6  <  1,  6  +  </  <  1,  has  proba¬ 
bility  function  /.,  (x,  ylc  =  0)  given  by  (23). 

Elementary  computations  provide  the  moments  of  the  BNBia.  b.  c.  p.  q.  v)  family.  In 
particular,  the  means  and  covariance  matrix  are  provided  in 

THEOREM  3;  The  mean  vector  and  covariance  matrix  for  the  BNBia,  b.  c.  p.  q.  v)  dis¬ 
tribution  are 


(24a) 

and 

(24b) 


IL  = 


1  = 


M  V 

vti  1 

> 

pO  2 

2 

P«,(l+«,) 

<r\  (Tx) 

'  '  \  -  d 

o-  u  o-^ 

~  |W;  P»2(l-t-»2) 

1  -  rf 

where  «i  and^<2  are  given  in  (8c). 


The  conditional  mean  of  Y  given  x  is  plotted  against  x  in  Figure  1 .  The  data,  taken  from 
Bates  and  Neyman  (31,  as  depicted  in  this  figure,  strongly  suggests  that  a  nonlinear  regression 
function  would  be  most  suitable  for  describing  the  empirical  relationship  between  X  and  Y. 
Unfortunately,  the  Bates-Neyman  distribution  does  not  admit  of  nonlinear  regressions.  The 
capability  of  allowing  for  nonlinear  regressions  seems  to  be  fundamental  in  data  analysis  situa¬ 
tions.  The  regression  function  for  BNBia,  b,  c,  p,  q,  1)  (and,  hence,  also  BNBia,  b. 
c,  p,  q.  m))  is  nonlinear. 


THEOREM  4:  The  regression  function 
distribution,  6  0,  is 


(25a) 

(25b) 


EiY\x)  -  -r^ 
1  -  q 


b  d  ^ 
b  m 


Ei  I'  l  x)  - 

l  -  q 


p  +  di]  -  p)x 
\-  di\-  p) 


EiY\x)  of  the 


b  ^  0, 


,  6  =  f  -  0. 


BNBia,  b,  c,  p,  q,  1) 
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But  (28)  may  be  expressed  in  terms  of  the  probability  generating  function  <^i(u.  v)  as 


(29)  #(z) -(!+»,) 

k~\ 

-  1  ^ 

1  ‘  II 

Az  z(z  ~  m) 

\l~  q 

[  m  +  6(1  -  p)  j  1 

z-k  (z-)t)(z-l) 

after  some  computation  and  simplification.  The  inversion  of  giz)  (Jury,  [7|)  gives  (25a),  an 
exponential  regression  function.  Letting  6  — *  0,  c  —  0  in  (2Sa)  gives  (25b). 

The  nonlinearity  of  regression  is  embodied  in  the  term  k'*'  in  (25a)  with  k  defined  in 
(26).  The  parameter  b  must  be  sizeable  for  small  m  in  order  that  the  regressions  be  nonlinear 
over  a  range  of  x  not  far  removed  from  zero.  Thus,  bivariate  negative  binomial  data  for  which 
this  form  of  regression  would  be  most  applicable  would  possess  appreciably  more  frequency 
counts  along  the  x-axis  than  along  the  v-axis. 


The  distribution  g,Ax.  >  )  is  central  in  the  discussions  of  all  the  probability  distributions 
discussed  to  this  point.  The  availability  of  an  easily  effected  computational  procedure  for 
g,.(x.  y)  would  facilitate  the  utilization  of  the  negative  binomial  distributions.  Since  g,.(x.  v)  is 
expressible  in  terms  of  the  hypergeometric  functions,  the  Gaussian  contiguous  functions  play  a 
dominant  role  in  any  computational  scheme.  First,  observe  from  the  definition  of 
F(a.  b.  c.  z)  that 


F(y  4-  X  z)  “  (1  —  z)'*'"'. 
F(y .  I'  +  y,  I-; z)  =  (1  —  z)""”'. 
Thus,  it  follows  immediately  that 


(30a) 


(30b) 


g^  (x  0)  ■ 
gJO.  y) 


(1  -</)(!-  p)(l  -  q) 

V 

P 

X 

g  +  X  -  1 

1  -  z 

1  -  z 

X 

(1  -</)(!-  p)(l  -  q) 

} 

V  +  y  —  1 

1  -  z 

1  -  z 

y 

The  use  of  relationship  15.2.18  of  Abramowitz  and  Stegun  [1]  allows  (15)  to  be  written  as 

1 


g^(x.  y)  =  (1  -  dY  hi{x)h2{y) 


x  +  V  + 


[xF  (i'  +  X  —  1 ,  «/  +  y ;  I' ;  z ) 


+  (p  +  y)  (1  —  z)  F(y  +  X,  f  +  y  +  1;  z))|. 

A  few  elementary  manipulations  give 

(31)  g^(x.  y  +  1)  »  7  .Twl - r  [(x +y +»')g^(x.  y) 

(y  +  l){l  -  z) 

-  pig  +  X  -  l)g„(x  -  1.  y)l. 

which  allows  the  entire  probability  distribution  to  be  computed  recursively  from  ^^(x.  0)  and 
g^(0.  y),  X  >  0,  y  ^  0. 
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Estimation  of  the  parameters  of  these  probability  distributions  generally  require  nonlinear 
optimization  routines.  Some  of  the  computational  burden  can  be  reduced  through  judicious  use 
of 


THEOREM  5:  For  the  BNBia.  0.  0.  p,  q.v)  distribution  and  if  i'  is  known  the  likelihood 
equations  for  a  random  sample  of  size  n  are 


/?  =  0 


where  L  is  the  likelihood  function,  x  and  y  are  the  sample  means  for  the  marginal  distributions. 


(32a) 

d\ogL 

da 

-  -Ey  -  R  =  0 
a 

(32b) 

aiogZ. 

dp 

{\  -  p)  -  .  _ 
- —  X  +  y  — 

P 

(32c) 

dlogL 

dq 

(1  -  q)y  .  - 

- ^  -E  y  — 

<7 

R  -  —  £  rt, 
n  fT 


V  +  1 


g^(x.  v  +  I) 
g,  .(x.  y) 


is  the  number  of  observations  for  which  X  ^  x. 


Y  »  y,  and  y)  is  the  probability  function  in  (15). 


PROOF:  If  the  probability  function  in  (15)  is  differentiated  with  respect  to  the  parame¬ 
ters  a,  p,  and  q  the  following  differential-difference  equations 


(33a) 

(33b) 

(33c) 


dg,(x,  y) 

(x.  v)- 

a 

3'  +  1 

gjx, 

da 

ad  d 

Str 

Q 

dg,.(x.  y) 

£  4- 

g,Ax.  y)  -  ■ 

1 

^  +  1 

dp 

1- 

P 

1  -  p 

d 

dg..(x.  y) 

y  1  -v 

g,.(x.  y)  -  ■ 

1 

y  +  1 

dq 

q  1  - 

q 

1  -  q 

Q 

g^(x,  y  +  1), 


(x.  y  -E  1). 


result;  these  equations  follow  by  using  (15.2.1,  Abramowitz  and  Stegun  [!)) 

3f(fl,  b,  c-,z)  ab  .  ,  .  ,  ,  ,  ,  ^ 

- - - =  —  F(a  -1-1.  6-(-l;c-l-l;  z) 

oz  f 


and  (exercise  1,  page  296,  Whittaker  and  Watson  (131),  namely 

f(a.  b  +  1;  c;  z)  —  F(a.  b\  c;  z)  =  —  F{a  -El.  6  -E  1;  c  -E  1;  z). 

c 

The  log  likelihood  function,  tog  L,  for  a  random  sample  of  size  n  is  ^  log  ^„(x.  y) 

JT.V 

and  so 

aiogZ,  _  _  1  Bg,Ax.  y) 

da  ^  g,.{x.  y)  da 

Using  (33a)  and  a  few  simple  operations  leads  to  (32a).  Similarly,  (32b)  and  (32c)  are 
obtained. 
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From  (32)  it  is  clear  that 

\  -  p)  -  il  -  q)  -  V 
P  q  a 

and  these  relationships  are  very  useful  in  estimating  the  parameters  via  the  method  of  max¬ 
imum  likelihood.  For  v  known  or  not,  the  conditions  in  (32)  are  necessary  for  a  maximum 
likelihood  solution  to  the  likelihood  function  for  ^  =  c  —  0.  Therefore,  (34)  can  be  used  to 
reduce  the  dimensionality  of  the  unknown  parameter  space  from  four  if  p  is  unknown,  to  two 
by  taking,  say,  p  —  qxl[qx  -I-  (1  -  q)y\  and  a  =  pql[(\  -  q)y\.  We  have  used  a  nonlinear 
optimization  computer  program  to  solve  for  the  parameter  estimates.  The  dimensionality 
reduction  permits  extremely  shorter  running  times. 

4.  APPLICATIONS  OF  THE  NEW  BNB  DISTRIBUTIONS 

This  paper  was  originally  motivated  in  part  by  the  visual  disagreeableness  of  the  fit  of  the 
regression  function 

£(K  lx)  - 

1  -  q 

of  the  Guldberg-Bates-Neyman  distribution  of  (S)  to  the  observed  data  as  depicted  in  Figure  1. 
The  maximum  likelihood  fit  of  the  Guldberg-Bates-Neyman  model  to  this  data  is  dismal  as 
measured  by  a  goodness-of-fit,  indeed,  a  linear  regression  is  clearly  not  appropriate  for  this 
data.  We  fitted  the  BNB  {a.  b,  c.  p.  q.  1)  and  the  BNB  (a,  b.  ().  p,  q.  p)  to  these  data 
(Bates-Neyman,  13])  but  obtained  i  -  c  »  0  in  the  first  case  and  6  =•  0  in  the  latter.  Even 
though  the  degree  of  fit  as  measured  by  increased  substantially,  the  overall  fit  was  still  very 
poor.  The  conclusion  that  the  Bates-Neyman  data  possesses  characteristics  which  preclude  the 
possibility  of  a  good  representation  by  a  bivariate  negative  binomial  seems  inescapable.  There 
is  thus  no  reason  to  present  any  of  our  results  concerning  this  data. 


TABLE  I  —  Bivariate  Data  Sets 


Description 
of  Data 

Sample  Size 
and 

Correlation' 

Marginals; 

Mean  and  Variance 

Parameters^  of 
Univariate 
Negative 
Binomial 
(EO.  (D)  iv.0) 

Fig  of  Uni¬ 
variate  Nega¬ 
tive  Binomial 
to  Marginals 
(x^  df.  P)^ 

Arbous-Sichel 

Absenteeism 

248,  0.73 

(X)  1947 

(Y)  1948 

4.70.18.66 

4.48.18.66 

1.58,2.99 

1.56,2.88 

8.9,  13,0.78 
11.0,12,0.53 

Bates-Neyman 

Diseases 

1286,  0.42 

(X)  Dig.  D. 

(Y)  Res.  D. 

1.40,  5.06 
5.32,22.13  1 

0.53,2.62 

1.69,3.16 

12.3,10,0.26 

25.4,20,0.20 

Aircraft 

Flight  Aborts 

109,-0.16 

(X) lst  6  mos 

(Y) 2nd  6  mos 

0.62,  1.03 
0.72,  1.08 

0.95,0.66 

1.51,0.48 

0.24,1,0.62 
6.3.  1,0.01 

' Pearson  product-momeni  ^Via  method  of  moments  ^Probability  of  exceeding  computed  value  with  indi¬ 
cated  degrees  of  freedom 


Table  1  shows  selected  summary  results  for  all  the  data  examined  in  detail.  The  first 
column  identifies  the  data,  column  2  provides  the  sample  size  and  correlation,  column  3 
specifies  the  marginal  random  variables  and  shows  the  associated  sample  means  and  variances. 


i 
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columns  4  and  S  show  the  parameters  of  the  univariate  negative  binomial  fitted  to  the  marginals 
and  the  associated  values,  degrees  of  freedom  and  probability  levels,  respectively.  Results 
from  this  table  will  be  presented  along  with  a  discussion  of  the  individual  data  sets.  The  absen¬ 
teeism  data  of  Arbous-Sichel  [2]  is  examined  first. 

From  Table  I  we  see  that  the  marginal  distributions  are  fitted  rather  nicely  by  the  univari¬ 
ate  negative  binomial.  Coupled  with  the  fairly  large  sample  correlation  coefficient,  it  seems  rea¬ 
sonable  to  expect  a  bnb  distribution  to  adequately  describe  the  data.  Arbous  and  Sichel  fit  the 
G  -  B  -  N(\,  0.  v)  model  to  their  data  and  report  a  of  17.0  on  13  degrees  of  freedom 
(F  =  0.20),  indicating  reasonable  agreement  of  the  data  with  the  model.  We  give  in  Table  2 
the  expected  cell  frequencies  obtained  from  the  BNB  (a,  0.  0,  p,  q,  v)  distribution  along  with 
the  observed  cell  frequencies  for  all  but  12  of  the  nonzero  observed  cells.  The  expected  cell 
frequencies  for  the  two  models  are  about  the  same  and  we  thus  expect  a  similar  probability  P  to 
obtain  for  the  fit  of  the  BNB  (a,  0.  0.  p.  q,  v).  Since  Arbous  and  Sichel  do  not  show  their 
grouping,  we  have  not  computed  the  x  ^  statistic.  The  fits  are  comparable  as  judged  by  cell  resi¬ 
duals. 


Although  the  fit  of  the  G  -  B  —  N  {\.  0 ,  v)  model  to  the  observed  data  is  reasonably 
good,  the  authors  point  out  that  12  of  the  18  observed  means  lie  below  the  theoretical  regres¬ 
sion  function  £(K|.v)  =  qip  +  x)/(\  -  q).  The  BNBia.  0.  0.  p,  q.  »/)  model,  via  maximum 
likelihood  estimates,  gives  rise  to  a  regression  function  for  which  only  10  of  the  18  observed 
means  are  less  than  the  predicted  values. 

An  attempt  to  fit  the  BNB  (a,  b,  c.  p,  q.  1)  and  BNB  (a,  b,  0,  p.  q,  p)  models  to  these 
data  was  made  but  we  obtained  maximum  likelihood  estimates  ^  -  c  -  0  in  the  first  case  and 
A  0  in  the  second.  The  lack  of  influence  of  the  parameters  indicates  the  apparent  adequacy 
of  linear  regressions  for  the  data.  Although  the  fit  of  BNBia,  0.  0.  p,  q,  v)  is  adequate,  a 
bivariate  beta  binomial  is  a  more  appropriate  distribution  for  this  data. 

Table  3  provides  observed  and  expected  cell  frequencies  for  flight  aborts  for  109  aircraft. 
The  fundamental  data  consist  of  109  pairs  of  observations  (xj.  >>/),  where  x,  represents  the 
number  of  aborts  by  aircraft  i  in  the  first  6  months  and  the  number  of  aborts  by  aircraft  /  in 
the  2nd  6  months  of  a  one  year  period.  Most  of  the  data  fall  in  the  (x,  0)  and  (0,  ,v)  cells  and 
this  implies  that  the  correlation  should  be  negative.  Note  that  the  probability  contours  are 
unlike  those  traditionally  associated  with  negative  correlation  and  that  the  distribution  has  its 
probability  concentrated  along  the  two  axes.  The  maximum  likelihood  estimates  for  the 
BNBia,  b,  c,  p,  q,  1)  model  are  o  -  0,  6  —  .68,  c  —  .32,  ^  -  .17,  q  —  .33.  These  values  pro¬ 
duce  an  estimate  of  correlation  of  -.13.  The  solid  borders  in  Table  3  indicate  the  grouping  we 
have  used  for  x^-  We  find  the  x^  statistic  to  be  10.2  on  6  degrees  of  freedom  or  F—  0.12,  a 
marginally  acceptable  fit.  The  observed  x  -l-  ^  has  frequencies  34,  37,  17,  10,  9,  2  for 

X  -t-  -  0.  1 . 5,  respectively,  and  0  for  all  x  -I-  >>  >  5.  The  mode  is  at  x  +  >»  -  1  which 

makes  more  plausible  a  BNB  ia.  b,  c.  p.  q.  p)  model  with  p  >  1  since  with  f  ~  1 ,  the  mode  is 
at  X  +  -  0.  Unfortunately,  the  BNBia.  b,  c.  p.  q,  1)  is  not  infinitely  divisible  (Block,  H.W., 

A.  S.  Paulson  and  R.C.  Kohberger  [4])  and  hence  we  will  not  be  able  to  produce  a  better  fit  to 
this  data  in  the  class  of  models  introduced  herein.  One  might  reasonably  be  hesitant  to  call  a 
distribution  which  is  not  infinitely  divisible  a  "true"  negative  binomial. 
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7  8  9  10  11  I  12 


7  7  8 

11.51  7.92  4.49  2.36  1.19  0.59  0.29 
9  6  8  7  13 

8.25  8.54  6.46  4.25  2.58  1.48  0.82 
9  9  6  5  13 

4.86  6.73  6.31  4.94  3.47  2.27  1.41 
17  5  4  3  2 

2.66  4.61  5.14  4.65  3.71  2.71  1.85 
17  5  4  3  2 

1.40  2.91  3.76  3.86  3.43  2.76  2.06 
2  1  1  2  5  2  1: 

0.72  1.74  2.56  2.93  2.87  2.52  2.04: 


0.36  1.01  1.66  2.09 
1 

0.18  0.57  1.03  1.42 
1 

0.09  0.31  0.62  0.93 

0.04  0.17  0.36  0.59 
I 

0.02  0.09  0.21  0.36 
0.01  0.05  0.12  0.22 
0.01  0.02  0.07  0.13 
0.01  0.04  0.08 
0.01  0.02  0.04 
0.01  0.02 
0.01  0.01 


2.24  2.13  1.85 
1  2  1 
1.65  1.69  1.57 
2  2 

1.16  1.27  1.26 
1  3 

0.79  0.92  0.97 

I 

0.52  0.64  0.71 
1 

0.33  0.43  0.51 
1 

0.21  0.29  0.35 
1 

0.13  0.19  0.24 
1 

0.08  0.12  0.16 
0.05  0.07  0.10 
0.03  0.04  0.07 
0.02  0.03  0.04 


0.14  0.06  0.03 

0.44  0.23  0.12 
2  1 
0.84  0.49  0.27 
I  1 

1.21  0.76  0.46 
1  I 

1.46  0.99  0.64 

1  1 
1.55  1.12  0.78 

2  2 
1.51  1.16  0.86 

3 

1.36  1.11  0.87 
2 

1.16  1.00  0.82 
2 

0.94  0.85  0.73 
1 

0.73  0.70  0.63 

0.55  0.55  0.51 
1  1 

0.40  0.42  0.41 
1 

0.28  0.31  0.31 

1 

0.20  0.22  0.24 

I 

0.13  0.16  0.17 
0.09  0.11  0.12 
0.06  0,07  0.09 


0.01  0.01 

0.06  0.03  0.02 

0.15  0.08  0.04 
1 

0.27  0.16  0.09 
1 

0.41  0.25  0.15 
3 

0.52  0.34  0.22  0.13 
1  1 

0.61  0.42  0.28  0.18 

0.65  0.47  0.33  0.22 
1  1 

0.64  0.48  0,35  0.25 
2 

0.60  0.47  0.36  0.27 
I  2 

0.54  0.44  0.35  0.27 
1 

0.46  0.39  0.32  0.26 
1 

0.38  0.34  0,29  0.24 
1111 

0.30  0.28  0.25  0.21 
1 

0.24  0.23  0,21  0.18 

0.18  0.18  0.17  0.15 
1 

0.13  0.13  0.13  0.12 
1 

0.10  0.10  0.10  0.10 
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TABLE  3  —  Observed  and  Expected^  Cell  Frequencies 
of  Flight  Aborts  for  109  Aircraft 


Isi  Six  Months 

'Bivariate  negative  binomial  distribution  of  (20).  Maximum  likeli¬ 
hood  parameter  estimates  are.  a  ”  0.  fr-  0.6820.  i  - 0.3179. 

0  I6.S.S.  V-  0.3299  There  results  x*  -  '0  2  and  P  -  0.12  for 
<//  -  6. 


5.  DISCUSSION 

We  have  produced  and  examined  properties  of  a  class  of  negative  binomial  distributions 
which  admit  of  nonlinear  regressions  and  negative  correlations.  The  most  useful  subset  of  this 
class  is  infinitely  divisible  and  therefore  allows  for  a  flexible  fit  to  data.  A  nonlinear  optimiza¬ 
tion  routine  was  used  to  produce  the  maximum  likelihood  parameter  estimates. 

Even  though  the  development  of  models  which  admitted  of  nonlinear  regressions  and 
negative  correlations  provided  some  of  the  impetus  to  this  work,  it  soon  became  clear  that 
bivariate  models  with  a  different  shape  parameter  for  each  margin  would  be  eminently  useful  in 
the  modeling  of  bivariate  data.  A  large  number  of  attempts  at  developing  such  appropriate 
negative  binomial  distributions  of  this  type  met  with  only  limited  success.  It  appears  that  a 
model  of  a  character  much  different  from  those  heretofore  discussed  in  the  literature  is 
required  to  produce  a  multivariate  negative  binomial  distribution  which  is  simultaneously 
infinitely  divisible,  allows  for  nonlinear  regressions,  and  different  marginal  shape  parameters. 
The  comparison  of  adequacy  of  agreement  between  data  and  model  was  based  on  the  usual  x  ’ 
goodness-of-fit  statistic,  a  procedure  with  serious  shortcomings.  An  alternative  procedure  for 
multivariate  goodness-of-fit  would  be  a  welcome  addition  to  the  multivariate  literature. 
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The  most  appealing  properties  of  our  BNB(a,  b,  0.  p,  q,  v)  and  BNBia.  0,  c,  p,  q.  v) 
distributions  are  their  infinite  divisibility  and  nonlinear  regressions.  However,  unless  nonlinear 
regressions  are  called  for,  it  appears  that  any  of  the  bivariate  negative  binomials  would  produce, 
via  maximum  likelihood,  an  acceptable  match  to  bivariate  negative  binomial  data.  It  should  be 
emphasized  that  the  maximum  likelihood  estimates  need  not  produce  the  best  fit  of  a  model  to 
the  data  as  measured,  for  example,  by  Further,  under  maximum  likelihood  and  ordinary 
least  squares,  each  observation  is  weighted  equally;  if  the  number  of  "taif  observations  relative 
to  the  complete  frequency  count  is  small,  then  these  procedures  need  not  produce  a  visibly 
nonlinear  regression  even  when  one  is  clearly  appropriate.  In  this  case  an  "anti-robusf  pro¬ 
cedure  may  be  preferable  to  maximum  likelihood  and  least  squares  in  order  that  legitimate  out¬ 
lying  observations  be  weighted  more  heavily.  A  case  in  point  is  provided  in  Figure  1  where  the 
bivariate  negative  binomial  model  is  not  appropriate  but  the  data  are  correct  and,  unfortunately, 
the  model  has  been  given  preference. 

Extension  of  our  results  to  dimensionality  in  excess  of  two  would  require  a  little  care  in 
ensuring  the  simultaneous  infinite  divisibility  and  nonlinear  regressions.  The  computational 
problems  would  increase  with  the  complexity  of  the  model  which  implies  a  necessity  for  parsi¬ 
mony  vis-a-vis  realism  tradeoff. 

Finally,  we  wish  to  emphasize  that  all  of  our  results  and  discussion  has  a  dual  counterpart 
in  the  bivariate  gamma  distributions  of  Paulson  (10].  Accordingly  we  would  define  and  work 
with  bivariate  gamma  distributions  flFGfa.  b.  0,  p,  q,  v)  and  BVGia,  0,  c,  p,  q.  v)  in  com¬ 
plete  analogy  with  the  preceeding  results. 
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ABSTRACT 

A  model  for  proper  scheduling  of  inspections  is  considered,  if  system 
failures  can  be  detected  only  by  checking.  Two  cases  are  analyzed:  replacement 
and  no  replacement  of  a  failed  system.  On  condition  that  no  or  only  partial  in¬ 
formation  on  the  lifetime  distribution  of  the  system  is  available,  minimax  in¬ 
spection  strategies  are  obtained  with  respect  to  cost  criierions. 


1.  INTRODUCTION 

The  following  situation  is  considered;  At  time  /  =»  0  a  system  starts  working.  The  time  to 
its  failure  (lifetime)  is  a  random  variable  X  with  the  probability  distribution  function  Fit), 
Fi+0)  -  0.  A  system  failure  is  assumed  to  be  known  only  by  inspecting.  Each  inspection 
entails  a  fixed  cost  ci  and  takes  only  negligible  time.  On  the  other  hand,  a  downtime  t  of  the 
system  (=  time  between  system  failure  and  its  detection)  gives  rise  to  cost  v(/),  where  v(/)  is 
a  continuous,  strictly  increasing  function  of  ^  0;  v(0)  =  0. 

Let  be  5  -  {f*l  an  inspection  strategy  (at  time  f*  the  ^h  inspection  takes  place,  when  no 
failure  has  been  detected  before,  0  «  /p  <  /|  <  fj  <  . . . ).  The  aim  of  this  paper  consists  in 
deriving  inspection  strategies,  which  are  optimal  with  respect  to  some  cost  criterions.  Results 
are  obtained  in  the  cases  where  the  lifetime  distribution  is  completely  or  partially  unknown.  In 
the  latter  case,  the  expected  system  lifetime  is  assumed  to  be  known. 

To  the  knowledge  of  the  author,  the  model  described  has  been  first  analyzed  by  Derman 
no]  (unknown  lifetime  distribution)  and  Barlow,  Hunter  and  Proschan  l2l  (known  lifetime  dis¬ 
tribution).  Modifications  of  the  basic  model  described  have  been  treated  e.g.,  in  (1,  3,  12,  14, 
15].  This  paper  summarizes  and  extends  results  obtained  by  the  author  [4,  S,  6,  8]. 

2.  INSPECTION  WITHOUT  REPLACEMENT 

If  an  inspection  strategy  (short:  strategy)  5  —  {f^}  is  in  effect,  then  the  expected  Total) 
loss  cost  KiS,  F)  up  to  detection  of  a  system  failure  amounts  to 

(1)  KiS.  f)  -  X  '  g,ii.  1,^0  dFit). 

k-o  ‘ 
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where 

gk(x.  y)  ^  (k  +  l)c|  +  vO»  -  x),  0  <  X  ^ 

If  a  downtime  cost  can  arise  only  in  the  finite  interval  iO,  T),  then  inspections  after  T are  not 
necessary.  Applying  in  this  case  a  strategy  S„  -  {4I  with  0-  fo  <  M  <  •  •  •  <  'n  <  i,i*\  T 
the  expected  loss  cost  is 

(2)  K(S„.  f)  -  ^  ft(/.  /*+,)df(/)  +  (m  +  l)c,(l  -  F(T)). 

Next  Fit)  is  assumed  to  be  entirely  unknown.  This  assumption  leads  to  the  cost  criterion 

(3)  KiS)  ^  sup  KiS.  F). 

Fit 

where  f  is  the  set  of  all  probability  distribution  functions  Fit)  with  F(+0)  =  0  and  KiS.  F)  is 
given  by  (2).  A  straightforward  estimation  of  KiS„,  F)  yields 

KiS„)~  max  gk(/*.  /k+i). 

*-().( . >1 

A  strategy,  SJ.,  satisfying 

KiS*.)  =  mm  KiS) 

is  called  "minimax  strategy." 

THEOREM  1 :  Let  n*  be  the  largest  integer  n  satisfying 

(4)  T  v-'ac,)  <  T, 

ti-O 

where  v"*(x)  is  the  inverse  function  of  \ii).  Then  there  exists  a  unique  solution  SJ-  -  {/•), 
0<  rr<  ^2  <  ■••  <  C-  <  T,of 

(5)  goiO,  i\)  -  gi(fi.  fj)  “  ...  -  g„-  (t„..  T) 
and  S*.  is  minimax  strategy. 

PROOF:  With  n  n*  Equation  (5)  is  equivalent  to 

(6)  v(6*)  “  v(8o)  -  kcj,  /c  “  0,  1 . n; 

8*  •“  7,  8*  “  f^+i  ~  th,  k  —  0.  I,  ....  n. 

k-O 

Thus,  in  view  of  the  definition  of  n*,  v(/*)  <  (n*  +  Dci  holds,  and  for  any  strategy  S,  with 
r  -  «•  +  1 

KiS*„.)  -  KiSr)  <  vfrf)  +  c,  -  (r  +  l)c,  <  vf/f)  -  in*  +  l)c,  <  0. 
Therefore,  a  strategy  S,  with  r  >  n*  cannot  be  minimax  strategy. 

The  functions  gkix.  y),  k  •*  0,  1 . have  two  obvious  properties; 

(i)  g*(x.  y)  is  strictly  increasing  in  y,  if  xis  fixed,  0  <  x  <  y. 

(ii)  g*  (x,  y)  is  strictly  decreasing  in  x,  if  y  is  fixed,  0  <  x  <  y. 
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Let  5'  -  {/;()  the  solution  of  (6)  with  0  <  w  <  w*.  Assume  there  exists  a  strategy  S„  -  |/*) 
satisfying 

(iii)  K(S„)  <  KiS'J. 

Let^  t,  ^  the  smallest  of  the  inspection  times  with  the  property  /,  pi  Because  of  (iii), 
/,)  <  K(S'„).  Taking  into  account  (i)  this  implies  /,  <  r,'.  Analogously,  (i)  and  (ii) 
yield  <  f,'+i  (for  otherwise  (iip  could  not  be  satisfied).  Continuing  in  this  way,  one  finally 
obtains  f„  <  and,  therefore,  g(t„.  T)  >  K(^)  (see  (16l).  Hence,  there  cannot  exist  a  stra¬ 
tegy  S„  satisfying  (iii).  If  and  0  <  nj  <  W2  ^  «*,  are  solutions  of  (6)  with  n  =  ni  and 
n  -  n2,  respectively,  then  it  can  be  proved  analogously  that  K(S'„)  <  This  completes 

the  proof  of  the  theorem. 

COROLLARY;  n*  is  the  largest  integer  n  for  which  there  exists  a  strategy  S'  =  {/*}  satis¬ 
fying  (6),  0  <  fi'  <  <  ...  <  t'„  <  C+i  =*  T.  Similarly,  t*  is  the  smallest  time  point  for  the 

first  inspection,  which  can  belong  to  a  solution  of  (6) . 

An  analogous  property  of  the  first  optimum  inspection  moment  with  respect  to  K  (S,  F) , 
if  fit)  -  F'{t)  is  known  and  of  type  PD2  (polya  density  function  of  order  2),  has  been  shown 
in  [51. 


Derman  IlO)  has  already  obtained  the  minimax  strategy  in  case  of  v(r)  =  C}!,  C2  >  0.  It 
is  given  by 


tt^k 


^  +4^(n*-k  +  ]) 


n*  +  1  2c2 


.  A  -  1.  2 . n*. 


where  n*  is  the  largest  integer  satisfying 

2c2T 


n(n  -E  1)  < 


Now  suppose  the  lifetime  distribution  of  the  system  to  be  unknown  with  the  exception  of 
the  expected  lifetime  fi  —  EiX),  0  <  /u,  <  <».  The  condition  that  downtime  cost  occurs  only 
in  (0,  T)  is  now  dropped.  Let 


KJS)  =  sup  K(S,  F). 


where  K  (S,  F)  is  given  by  ( I )  and 

,dFU) 


F,- 


A  strategy,  S*,  satisfying 

K^(Sl)  -  min  K^iS) 

is  called  "partial  minimax  strategy." 


Within  a  more  general  model  Hoeffding  111)  has  shown  that  for  any  S  -  It*} 
sup  KiS,  T)  -  sup  KiS,  F), 

rtf;-'- 
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where  is  the  subset  of  those  elements  of  having  exactly  two  points  of  increase.  Routine 
calculations  yield 


(7) 

where 


KAS)~  sup  GAS). 

^  0</^m  ^ 

m<j 


and  m 
(8) 


V  V  ** 

m(S)  is  defined  by 

<  #*  <  'm+i  (see  [81  for  details). 


A  strategy,  5  -  is  called  strictly  periodic  with  the  inspection  interval  8,  if 
8  -  8jt  -  /*+!  -  for  all  fc  «  0,  I ,  ...  ;  notation;  S  -  S**’.  K^(S)  has  a  simple  structure  for 
strictly  periodic  strategies; 

(9)  A:^(S<«')  =  yM  +  v(8)  +  c,. 

This  results  also  from  a  straightforward  estimation  of  K  (S'*’,  f )  if  F  €  F^; 

/•(/{  + 1)8 

a: (S'*’,  F)  «  T  I,  u*  +  1)C|  +  V((A  +  1)8  -  f)l  dF(t) 

<  H  J..  Kfc  -»■  dFit) 

*-0-^** 

“  r  (*S)  [F((A  +  1)8)  -  F(it6)l  +  v(8)  +  c, 

°  *-o 

^  +  v(8)  +  C|. 

o 

Thus,  if  v(r)  is  differentiable  the  optimum  inspection  interval  8*  with  respect  to  AT^fS'®’) 
satisfies 

(10)  8^  v'(8) 

There  exists  always  a  positive  solution  of  this  equation.  Especially,  for  v(/)  —  C2t  the  unique 
solution  is 

(11)  8  *  *  -JficJ Cj. 


THEOREM  2;  There  exists  a  strictly  periodic  partial  minimax  strategy. 


PROOF;  Assuming  there  exists  a  strategy  S  —  (r*)  with  the  property 
(12)  A/S)  <  A'^IS**^’). 

G,j(S)  can  be  written  in  the  form 


with 


G,j(S) 


(/■  +  l)ci  +  v(8j)  +  (fi  —  ii)c\  Oij 
(J  -  /)C|  +  v(dj)  -  v(8,) 

(fj  -  Oci 


0  ^  i  ^  m  <  J. 
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Further,  let  a,  -  sup  a„  and  a  =  min  o,.  Without  loss  of  generality  it  can  be  assumed 

j>m  ■'  0<i<m 

0  <  a  <  oo.  Putting  8  -  1/a  there  holds  in  view  of  (9)  and  (12)  for  all  0  <  i  <  m 

(/■  +  l)f|  +  v(8,)  +  (/t  —  t)c\a  <  fic^a  + 

Starting  with  i  0  induction  yields 

(13)  8,  <  1/a,  0  <  /  <  m. 

Let  be  /'o  defined  by  =  a.  Then  it  must  be 

(J  -  /o)ci  +  v(8y)  -  (8,,,) 

- - - r - ^  a.  j  >  m. 

^1,  -  •,) 

Using  (13)  it  can  be  inductively  seen  that  hj  <  8,^,,  j  >  m.  But  this  implies 
a  =  sup  a,  .  >  1/8,  ,  contradictory  to  (13).  Hence,  there  cannot  exist  a  strategy  S  satisfying 

i>m  " 

(12)  and  the  theorem  is  proved. 

The  proof  given  shows,  moreover,  that  every  partial  minimax  strategy  must  be  strictly 
periodic.  Its  inspection  interval  is  a  solution  of  Equation  (10). 


Analogous  problems,  where,  instead  of  the  expected  lifetime,  a  percentile  of  the  lifetime 
distribution  is  assumed  to  be  known,  have  been  treated  in  15,  8,  13,  151. 


3.  INSPECTION  AND  REPLACEMENT 


In  this  section  an  obvious  extension  of  the  basic  model  with  fi  known  is  considered. 
Immediately  on  discovery  of  a  failure  the  system  is  replaced  by  a  statistically  equivalent  one. 
Each  replacement  requires  a  fixed  cost  c ,  and  a  fixed  lime  d.  The  inspeciion»replacemeni  pro¬ 
cess  is  continued  unlimitedly. 


In  what  follows  it  is  assumed  v(r)  -  C2/,  where  ci  is  restricted  by 
(14)  (c,  -F  c,)/#i  <  C2. 

Otherwise,  the  expected  loss  cost  per  unit  downtime  of  the  system  would  be  smaller  or  equal 
than  the  mean  loss  per  unit  time  which  arises  by  "ideal  inspection  and  replacement"  (i.e.,  failure 
of  the  system  is  detected  immediately  and  replacement  takes  negligible  time).  But  then  inspec¬ 
tion  and  replacement  are  uneconomical  from  the  first. 


The  expected  length  of  a  cycle  (time  between  two  neighboring  replacements)  by  applica¬ 
tion  of  the  strategy  S  «  (t*)  is 

LiS.  f)  =  £  lf(/*+,)  -  f(r,)l  +  d, 

k~Q 


Thus,  the  expected  loss  cost  per  unit  time  C(S,  F)  amounts  to 


C(S,  F) 


K(S.  F)  +  cj 
US.  F) 


where  K(S,  F)  is  given  by  (1). 
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Brender  [9]  has  dealt  with  the  problem  of  calculating  an  optimum  inspection  strategy  with 
respect  to  C(S.  F)  if  F  is  known.  If  F  is  completely  unknown  the  corresponding  mimimax 
inspection  strategy  is  easily  seen  to  be  trivial— no  inspection  and  replacement  at  all  (a  nontrivial 
minimax  inspection  strategy  exists  if  v(r)  is  strictly  convex  l7]). 


Let 


=  su^ 


C(S.  F). 


To  get  the  corresponding  partial  minimax  strategy,  the  following  lemma  is  needed.  For  being 
only  a  slight  generalization  of  an  approach  used  by  Brender  [9]  its  straightforward  proof  can  be 
omitted  (see  16]  for  details). 


LEMMA;  Let  be 

Q(S.  x)  =  sup  [KiS.  F)  +  c,-  xL(S.  F)] 

and  assume  for  all  x,  0  <  x  <  ci,  the  existence  of  a  unique  strategy  S(x)  and  of  a  number  Xo, 
0  <  X()  <  Cl,  so  that 

Q(.S(x),  x)  =  mm  QiS,  x)  and  ^(S(xo),  xo)  =  0. 

Then  5(xo)  is  the  unique  partial  minimax  strategy  and  x,)  =  C^iSixn)). 

KiS,  F)  -  xUS,  F)  has  for  every  x,  0  <  x  <  c^,  the  same  functional  structure  as 
KiS,  F).  Hence,  Theorem  2  secures  the  existence  of  a  strictly  periodic  strategy  S(x). 
According  to  (11)  its  inspection  interval  Six)  is  given  by 

(15)  8(x)  =  ■Jnc\/ic2  -  x). 

It  follows  from  (9)  and  (15) 

0(S(x).  x)  =  2  -JfiCiic2  -  x)  -  xipi  +  d)  +  C)  +  cjd  +  c,. 

Of  course,  QiSiO),  0)  >  0,  and  in  virtue  of  (14)  ^(5(cj),  C2)  <  0.  Hence,  a  unique  solu¬ 
tion  Xo  of  0(S(x).  x)  =  0  exists; 


1 

- 

H  +  a 


C2M  -  c,  -  c,  + 


fi  +  d 


Minimax  inspection  problems  for  replaceable  systems,  where,  instead  of  the  expected  life¬ 
time,  a  percentile  of  the  lifetime  distribution  is  known,  have  been  treated  in  (6,  16). 
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ABSTRACT 

In  ihis  paper  the  rcliabiliiv  function  R  =  P(  \  <  K)  has  been  estimated 
when  V  and  K  follow  gamma,  exponential  or  bivariate  exponential  distributions 
The  paper  ts  p;irlly  expository. 


1.  INTRODUCTION 

Let  X  and  Y  be  two  random  variables  with  cumulative  distribution  functions  Fix)  and 
G(>)  respectively.  Suppose  Y  is  the  strength  of  a  component  subject  to  a  stress  X.  Then  the 
component  fails  if  at  any  moment  the  applied  stress  (or  load)  is  greater  than  its  strength.  The 
stress  is  a  function  of  the  environment  to  which  the  component  is  subjected.  Strength  depends 
on  material  properties,  manufacturing  procedures,  and  so  on.  The  reliability  of  a  component  is 
the  probability  that  its  strength  exceeds  the  stress.  From  practical  considerations  it  is  desirable 
to  draw  inference  about  the  reliability  function. 

The  above  model  was  first  considered  by  Birnbaum  (2l  and  has  since  found  an  increasing 
number  of  applications  in  many  different  areas,  especially  in  the  structural  and  aircraft  indus¬ 
tries.  For  a  bibliography  of  available  results  see  Basu  [1]. 

In  many  situations,  the  distribution  of  X  (or  of  both  X  and  F)  will  be  completely  known 
except  possibly  for  a  few  unknown  parameters  and  it  is  desired  to  obtain  parametric  solutions. 
Thus,  in  case  of  missile  flights,  the  stress  may  be  expensive  to  sample,  but  the  physical  charac¬ 
teristics  of  the  missile  system,  such  as  the  propulsive  force,  angle  of  elevation,  changes  in 
atmospheric  condition,  and  so  on,  may  all  have  known  distributions;  consequently,  the  distribu¬ 
tion  of  stresses  can  be  calculated.  Church  and  Harris  [4],  Owen,  Craswell  and  Hanson  (81,  and 
Govindarajulu  l6l  have  considered  the  above  problem  under  the  assumption  that  X  and  Y  have 
normal  distributions.  Since  in  many  physical  situations,  especially  in  reliability,  exponential  and 
other  distributions  provide  more  realistic  models,  it  is  desirable  to  obtain  estimators  of  R  for 
distributions  useful  in  life  testing.  In  section  2  we  consider  gamma  and  exponential  distribu¬ 
tions  under  the  assumption  that  X  and  Y  are  independently  distributed.  The  case  of  a  bivariate 
exponential  distribution  is  studied  in  section  3.  Two  distribution  free  procedures  are  mentioned 
in  section  4.  The  effect  of  misspecifying  the  model  is  considered  in  section  S  and  a  numerical 
example  is  given  in  section  6. 


■  this  rcso.irth  wax  suppciriyd  in  pari  under  toniracl  NOO()l4-7lt-C-Oh55  lor  the  Office  of  Naval  Research. 
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2.  AND  EXPONENTIAL  DISTRIBUTION 


Let  Land  )  be  independently  distributed  with  density  functions 


12  1) 
12. 2t 


/(v)=  - 


,e(  .t  I  = 


!'(/)) 


!(</) 


../■  I 


.V  >  0.  /)  >  0 


<'  y  >  0.  >  0 


respectively.  Then 

(2  3)  =  /’(.V  <  y 


_  -  ^  -  ^ 

a'' 

Jo  [Jv  r(</) 

Yip) 

y'  Yip  +  k) 

a''l3‘ 

^4r,  r(/>)r(/(  -p  1) 

(a  -P/3)'’"^' 

c  •"x'’  -'dx 


Here  p  and  q  are  assumed  to  be  known  integers. 


Note;  /?||  =  «/(a  +li).  and  /?,,(  =  Rfi  for  all  p.  Also, 

/?,„=  X  ^  Z  *'  - 

*  II  H  ^,(1 

In  particular, 

R2\  =  R  ri.  R\[  =  R  j'|. 

^  i:  =  ^11  +  03/« )  R  u 

ft,,  +  {fi/a)Ri,  +  (/3/a)ftj',. 
ft::  =  ft,-,  (3  -  2ft,,). 

If  ft,,  is  close  enough  to  one,  as  is  expected  for  items  with  high  reliability, 

(I  -  «,i). 

Expressions  for  ft,,,  and  ft,„  indicate  that  in  this  case  the  expression  for  the  reliability  is  not 
strongly  dependent  on  the  choice  of  the  parameter  p  (especially  if  p  is  small)  and  the  distribu¬ 
tion  of  X  can  be  approximated  by  the  exponential  distribution  without  much  distortion  in  the 
value  of  ft,,,.  However,  so  far  as  the  parameter  </  is  concerned,  the  situation  is  quite  reversed. 
The  value  of  reliability  is  heavily  dependent  on  the  choice  of  the  underlying  distribution  of  Y 
and  one  has  to  choose  the  value  of  q  more  carefully.  Later  in  section  5,  we  shall  further  study 
the  effect  of  misspecifying  the  parameters  p  and  q  and  confirm  our  conclusions  by  numerical 
studies. 


If  two  independent  random  samples,  (T,.  Ti . X,„)  and  fS .  Y„),  from  the 

two  gamma  populations  are  available,  maximum  likelihood  estimators  (m.l.e.)  of  a  and  /3  are 

given  by  «  =  -=  and /d  =  Hence,  m.l.e.  of  ft,„,  is 
X  y 


(2.4) 


A  ^  y  r(p  -h  k)  «'’/§* 
*4:,  ripH'lA  -PI) 


As  special  cases,  if  </  =  1,  that  is,  if  X  follows  the  gamma  distribution  and  Y  follows  the 
exponential  distribution. 
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Finally,  if  both  p  and  q  are  equal  to  I,  we  have  the  case  of  two  independent  exponential  distri¬ 
butions,  and  we  have 

(2.6)  Ru - 

a  -1-/3  X  +  Y 

X 

Since  R  in  2.6  is  a  function  of  the  exact  distribution  R  can  be  obtained  in  the  exponential 

X  a 

case.  It  is  well  known  that  —  —  follows  the  /'distribution  with  (2m,  2n)  degrees  of  freedom. 

.  Y  /3 

Thus,  the  distribution  of  /?||  follows.  The  result  will  be  used  later  in  section  5  to  compare  the 
performance  of  independent  exponentials  with  those  of  dependent  exponential  models.  Using  a 
theorem  in  Rao  [101,  Theorem  6a.2,  page  321,  the  distribution  of  in  each  of  the  above 
cases,  for  large  m  and  n,  can  be  shown  to  be  normal  and  hence  an  estimate  of  the  asymptotic- 
confidence  interval  for  R  can  be  obtained.  Thus,  in  this  case  with  m  =  n  we  have  for  large  n, 
\/n  (R,,^  —  Rp^)  ~  Expressions  for  and  for  a  few  selected  values  of  p 

and  q  are  given  below. 

»  _  s _ 1— 


a 


■  o  7 

.  R  21  ~  =  =  r .  fr-  21 

(Ar-h2T)- 


(a  -1-/3)*' 


n  ^  _ 

a  + 13  (a  + 


R  =  y  _  +  2A'T 
(P-i-2;r)  (Y  +  2X)^ 


0-2  =  6a  ^/3^ 

(a+/8)*'' 


R  =  I  ^  ^  Y^  2XY^ 

’■  (a  -4-/3)^  (a-f/S)’’  (A” -f  T)^  (P -1-  K)' ' 

^u  —  36(a/3)V(a  +P)*. 

3.  BIVARIATE  EXPONENTIAL  DISTRIBUTION 

Since  the  exponential  distribution  is  considered  a  useful  model  in  life  testing  problems,  it 
is  desirable  to  consider  bivariate  analogues  of  univariate  exponential  distributions  which  will 
have  properties  similar  to  those  of  the  univariate  exponential  distribution.  Marshall  and  Olkin 
(7)  have  proposed  a  very  important  bivariate  exponential  distribution  (BVE),  which  is  given  by 


Fix.  .v)  =  PiX  >  X.  T  >  v)  =  p 


\  j  v-A  2'  -A  I  ymaxf  v.i ) 


0  <  \  \  X,2  <  oo,  \  ,  >  0,  \  2  +^I2  >  0(x  >  0.  y  >  0). 

The  BVE  arises  in  several  natural  ways  and  is  considered  a  useful  model  in  reliability  with 
appealing  properties. 

Let  (AT,,  Y,),  /  ”  1.  2 . n  be  a  random  sample  from  (3.1).  We  shall  estimate 

P(X  <  Y)  when  (X,  Y)  follows  the  BVE.  It  can  be  readily  obtained  from  (3.1)  that 


R  =  PiX  <  Y)  = 


X  I  -1-  X  1  -(-  X  1 
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Hence,  R  is  estimated  by 

(3.3)  R  -  ^ 

Xl  +X2  +^I2 

where  X],  X2,  and  X12  are  the  maximum  likelihood  estimators  of  X|,  X2,  and  X|2,  respectively. 
Various  authors  have  considered  maximum  likelihood  estimation  of  the  parameters  X  X2,  X|2. 
However,  no  explicit  solutions  of  these  parameters  are  available.  In  order  to  obtain  an  explicit 
form  for  R,  we  replace  the  m.I.e.  by  some  special  ad  hoc  estimators  called  the  ’INT  estimators 
of  Proschan  and  Sullo  [9]  which  have  very  high  asymptotic  relative  efficiency  compared  with 
the  m.I.e.  estimators.  Let  n,  -  number  of  pairs  such  that  A',  <  Lj,  n2”  number  of  pairs  with 
Xi  >  Y,  and  hq  —  numbers  of  pairs  with  —  Y,. 

The  "INT  estimators  are  given  by 

0,4. 


«l 

"  x*  - 

»2 

n 

«i  +  no 

1.x. 

"2  +  "0 

n 

1  J', 

=  «0  1  +  .  +  . 

[  rt|  +  /lo  n?  +  Wo 

Proschan  and  Sullo  also  prove  the  following  theorem. 


^  max(3fi,  y,). 


THEOREM  (Proschan  and  Sullo):  n'^^(X,  -  X)  is  asympoto,'cally  trivariate  normal  with 
mean  0  and  dispersion  matrix  I  -  (cr,y)  where,  using  the  suffix  n  to  denote  dependence  on  the 
sample  size,  X,  =  (Xi,  X2,  X12)',  X  =  (X|,  X2,  X12)',  0  =  (0,0,0)' 

(3.5)  (r,|  “  (X|(X  -  XiXjyf'), 

o- 13  =  —  X  1X2X  i2(X  iX2y2  ^  +  Xy  f')/(Bo>'|). 

^■22  “  ^2(^  ~  ^  1^2y2  ')> 

O’ 23  =  —  X  |X  3X  |2(X  |X2y  1  *  +  Xy2  ')/(0oy2)> 

O’ .33=  ^I2(^  -  ^(^(r^(^iy2''^  +  ^2’>'r^  +  2.*- |X2yrV2'^)J. 
and 

X  ”X|  +  X2  +  X12,  yi“X|  +  X12, 
y2  -  X2  +  X  12 

#0 ”  £^lmax(3f,  y)l  =  yf' +  y2“' -  X^'. 

•  a  A  A  A  «.  A 

R  is  then  estimated  by  R  -  X  ||(X  1  +  X2  +  X 12)  rather  than  by  R.  Since  R  is  a^  totally 
dilTerential  function  of  X  |,  X2,  and  X  12  by  Theorem  6a.2(ii),  page  321  of  Rao  [10],  >/»  (R  —  R) 
is  asympototically  normally  distributed  with  mean  zero  and  variance  o-^  where 

(3.6)  o”'-  —  O’,,  -f  —  (0-22  +0-33)  -  2  (O’, 2  +0- 13)  +  2  -i-o’23. 

A  A  A  A 
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Thus,  from  (3.3),  (3.4)  and  (3.6).  one-sided  and  two-sided  confidence  limits  for  R  can 
readily  be  obtained. 

To  check  the  adequacy  of  the  large  sample  approximation  when  the  underlying  distribu¬ 
tion  is  BVE,  computer  simulations  were  made.  For  various  values  of  (X|,  X2,  X12)  SOO  sets  of 
random  samples  of  size  n  (n  •  10,  15.  20)  from  bivariate  exponential  distributions  were 
obtained  and  the  empirical  distribution  of  R  was  obtained.  Even  for  sample  size  as  low  as  10 
the  exact  distribution  is  found  to  be  well  approximated  by  the  normal  approximation.  The 
situation  improves  as  the  sample  size  increases. 

4.  DISTRIBUTION  FREE  PROCEDURES 

In  all  the  cases  considered  in  the  previous  section  to  check  adequacy  of  normal  approxi¬ 
mation,  the  value  of  R  is  rather  small  (R  <  .75),  whereas  applications  of  interest  would  be  for 
systems  with  high  reliability  (R  >  .90).  Unfortunately,  in  all  the  cases  with  R  >  .90  con¬ 
sidered,  the  sample  estimate  of  the  variance  of  R  came  out  to  the  negative  or  very  close  to 
zero.  (The  situation  is  similar  to  the  problem  of  having  ’negative*  estimator  of  variances  in 
analysis  of  variance  problems.) 

To  study  the  cases  for  which  R  >  .90,  we  therefore  consider  the  following  estimators  of 
R: 


For  bivariate  data,  a  rather  natural  way  to  estimate  R  —  P(X  <  Y)  would  be  based  on 
the  binomial  distribution.  Let  Tbe  the  number  of  cases  for  which  X  <  Y.  Then  7"  is  a  bino¬ 
mial  random  variable  with  mean  R  and  variance  nR(\  -  R).  We  therefore  can  obtain  an  exact 
binomial  confidence  interval  or  an  approximation  two-sided  100  (1  -  >)%  confidence  interval 
given  by 


(4.1) 


R  +  Z, 


^R(l  -  r1 


where  R  =  Tin  and  Zy  is  such  that  <t>{Zy)  -  (1  -  y/2),  where  <!»(•)  is  the  standard  normal 
distribution. 


If  X  and  Y  are  assumed  independent,  a  second  estimator  is  the  following  nonparametric 
confidence  interval  proposed  by  Govindarajulu  l6l  and  is  based  on  the  Wilcoxon-Mann- 
Whitney  statistic. 

Let  (Jf),  X2 . X„)  and  (K|.  Y2 . Y„)  be  two  independent  samples  of  measure¬ 

ments  from  populations  with  distribution  functions  Fix)  and  Giy)  respectively.  Let 

jl  if  3r,  <  Y, 
i/iiX,,  T,)  -  |q  otherwise 

m  n 

then  U  -  ^  5^  i|>(.K,,  Y,)  is  the  well-known  two  sample  Mann- Whitney  statistic.  That  is, 

I- I  ;-l 

U  -  number  of  pairs  iX,.  Y,)  such  that  X,  <  Y,.  Govindarajulu  (61  has  explicitly  derived 
one-sided  and  two-sided  distribution  free  confidence  bounds  for  R  (actually  Govindarsyulu 
derived  confidence  bounds  for  1  -  R)  based  on  the  asymptotic  normality  of  R  -  Ulmn.  In 
particular,  for  the  two-sided  case,  Govindarajulu  showed  that  for  all  Rand  G  and  large  m  or  n, 
the  solution  of  the  inequality 
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H(\R  -  R\  ^  >  \  ~  y.  0<y<l 

IN  ifir  cn  h> 

»,  ^  (4*-)  '  ■«t>  '(1  -  y/2). 

Mere  r  >  min(m.  n)  and  <t>(  )  is  the  cumulative  distribution  function  of  standard  normal  distri¬ 
bution  In  particular,  if  m  ~  /t.  a  100  (I  -  >)%  confidence  interval  is  given  by 

(4  y)  (R  -  R  +«^). 

where  R  and  Z.  are  as  defined  before,  and 

IsfJ,' 


5.  EFFECT  OF  MISSPECIFYING  THE  MODEL 

In  section  2  it  was  pointed  out  that  for  value  of  PiX  <  Y)  close  to  unity,  Is  less  sen¬ 
sitive  to  the  variation  of  p  and  varies  considerably  for  varying  values  of  q.  To  study  the  effect 
of  misspecifying  the  model  we  carry  out  the  following  Monte  Carlo  experiment. 

Let  G(<x.  p)  and  (71/3.  q)  be  two  given  gamma  populations  with  known  parameters 
(«,  p)  and  (fl.  q)  respectively,  where  o  =»  19  and/3  =  1.  In  this  case,  if  p  “  ^  —  1,  Rii  ==  .95. 
By  choosing  different  pairs,  ip.  q),  we  get  different  pairs  of  gamma  distributions. 

Let  (T|,  X2 . X„)  and  (K|.  K2 . Y„)  be  two  random  samples  from  G(qi,  p\)  and 

G(fi.  q[).  Since  p\  and  ?i,  the  true  values  are  not  known,  there  is  a  possibility  that  we  will 
choose  a  different  pair  of  distributions  as  the  true  model.  Let  us  assume  that  the  above  sam¬ 
ples  have  come  from  populations  with  distributions  G(o.  pj)  and  G(/3.  </j).  Thus,  we  would 
estimate  instead  of  estimating  the  true  value  and  compute  a  confidence  interval 

based  on  Rp,^,.  We  would  not  commit  much  specification  error  if  this  confidence  interval  con¬ 
tains  the  true  unknown  value  For  a  given  n  and  two  pairs  of  values  (p|,  qi)  and 

(p>.  qj)  the  above  procedure  is  repealed  1000  times  and  a  count  is  made  of  how  many  times 
the  true  value  is  contained  in  the  confidence  interval  based  on  Rp^^p  We  repeat  the  pro¬ 
cedure  for  different  values  of  n  and  different  combinations  of  (p\,  qi)  and  (p2,  (?2)-  The 
results  are  given  in  Table  1.  Here  n  is  chosen  to  be  S,  10  and  25.  All  combinations  of  the  fol¬ 
lowing  pairs  of  values  are  chosen  for  (p|,  q,)  and  (pi.  Qi)'.  (I,  I),  (1,  2),  (2,  1)  and  (2,  2). 

From  Table  I  we  can  make  a  number  of  conclusions.  First,  note  that  no  parametric 
method  performs  well  in  all  situations.  For  all  the  gamma  models  considered,  the  procedure,  as 
anticipated  in  section  2,  is  robust  for  small  variation  in  p.  However,  it  is  sensitive  to  variation 
in  q.  The  nonparametric  confidence  intervals  based  on  the  Wilcoxon-Mann-Whitney  statistic 
performs  well  in  all  cases.  However,  in  each  case  the  width  of  the  confidence  interval  for  an 
assumed  model  is  too  large  compared  with  the  corresponding  width  based  on  the  parametric 
models.  In  each  case  the  parametric  method  is  to  be  preferred,  especially  if  the  parameter  q  is 
reasonably  well  specified. 
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TABLE  1  —  a.  Number  of  Counts  for  N  =  5 


ASSUMED  MODEL 

TRUE  MODEL 

Gd.  1)  =  (.(2.  1)  Gd.  2)  G(2.  2) 

Exponential 

^/’|V| 

.95 

.9025 

.9975 

.99275 

Gd.  1) 
exact 

933 

967 

46 

77 

Gd.  1) 

normal 

approx. 

855 

934 

1000 

1000 

G(2.  1) 

763 

819 

66 

94 

Gd.  2) 

287 

487 

828 

938 

G(2.  2) 

209 

371 

725 

871 

*NP  pr 

998 

991 

1000 

1000 

‘Ndnp.ir.iiiiciric  Rrocoilurc 


TABLE  1  —  b.  Number  of  Counts  for  N  —10 


ASSUMED  MODEL 

TRUE  MODEL 

Gd.  1)  =  1)  Gd.  2)  G(2.  2) 

Exponential 

.95 

.9025 

.9975 

.99275 

Gd.  1) 
exact 

947 

976 

0 

2 

Gd.  1) 

normal 

approx. 

913 

952 

188 

348 

G(2,  1) 

775 

838 

0 

1 

Gd.  2) 

133 

293 

869 

968 

G(2.  2) 

58 

192 

750 

906 

*NP  pr 

999 

996 

1000 

1000 

*Nonpiir;imelric  Procedure 
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TABLE  1  —  c.  Number  of  Counts  for  N  ”  25 


ASSUMED  MODEL 

TRUE  MODEL 

C(l.  I)=  G(2.  1)  Gd.  2)  G(2.  2) 

Exponential 

.95 

.9025 

.9975 

.99275 

G(l.  1) 
exact 

945 

964 

0 

0 

C(I.  I) 

normal 

approx. 

934 

964 

0 

0 

G(2.  1) 

768 

812 

0 

0 

C(l.  2) 

6 

82 

925 

982 

G(2.  2) 

1 

27 

754 

928 

•NP  pr 

1000 

998 

1000 

999 

*Nonparametric  Procedure 


6.  AN  EXAMPLE 

To  illustrate  the  computation  of  confidence  intervals  let  us  consider  the  following  exam¬ 
ple.  Fifteen  items  of  random  strengths  K| .  Kjs  are  subject  to  random  stresses 

X,,  Xi . -Vi,.  To  estimate  the  reliability  function  P(X  <  Y)  random  samples  of  15  pairs 

of  (T,  Y)  values  are  drawn  and  given  below. 

Pair  No. 


1 

.0352 

1.7700 

2 

.0397 

.9457 

3 

.0677 

1.8985 

4 

.0233 

2.6121 

5 

.0873 

1.0929 

6 

.1156 

.0362 

7 

.0286 

1.0615 

8 

.0200 

2.3895 

9 

.0793 

.0982 

10 

.0072 

.7971 

11 

.0245 

.8316 

12 

.0251 

3.2304 

13 

.0469 

.4373 

14 

.0838 

2.5648 

15 

.0796 

.6377 
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From  past  record  it  is  known  that  R  -  .95.  Estimates  of  and  corresponding  confidence 
interval  will  depend  on  the  model  chosen  and  the  method  used  to  compute  the  confidence 
interval.  Thus,  if  X  and  Y  are  -Tssumed  independent,  we  have  two  independent  samples  of  size 
15  each.  Table  2  lists  the  values  of  R^^^^  and  confidence  interval  for  the  cases  considered  in 
section  5.  We  also  use  the  notation  of  section  5  for  convenience. 


TABLE  2  —  Comparison  of  Various  Confidence  Intervals 
and  Estimates  of  R 


Model  Used 

- :: - 1 

R 

Confidence  Interval 

C(l.  1)  exact 

.9639 

(.9280,  .9823) 

C7(I,  1)  normal  approximation 

.9639 

(.9300,  .9978) 

G(2,  1) 

.9639 

(.9416,  .9856) 

G(l.  2) 

.9952 

(.9896.  1.000*) 

G(2,  2) 

(.9925,  .9999) 

Binomial  Procedure 

(.8071,  1.000*) 

NP  Procedure 

.9600 

(.7070,  1.000*) 

as  bcine  larger  than  I 


_  Let  us  illustrate  the  calculation  for  a  couple  of  cases.  From  the  above  data  we  have 
X  "  .0509  and  ?  =  1.3602.  If  we  assume  pj  =  =  1  we  have,  using  results  in  section  2, 

/?,,  =  :=  ---  “  .9639 
AT  +  K 

and  corresponding  95%  confidence  interval  for  using  normal  approximation,  is  (.9300, 
.9978). 

On  the  other  hand,  if  we  used  the  exact  distribution  of  obtain  the  required 

y  B 

confidence  interval  from  an  F-table  since  U  =  ~  —  follows  the  F-distribuiion  with  (2n,  2n) 

X  a 

degrees  of  freedom  we  obtain,  after  some  simplification  (.9280,  .9823)  as  the  required  95% 
confidence  interval. 

From  Table  2  it  seems  any  of  the  first  three  procedures  based  on  two  independent 
exponential  distributions  or  C(2.  1)  (AT gamma  with  shape  parameter  p  -  2  and  K independent 
exponential)  will  be  quite  satisfactory. 
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ABSTRACT 

A  Corni  of  soqueniial  decision  problem  is  iniroduced  in  which  options  are 
presented  in  sequence,  with  no  recall  of  rejected  options  (as  in  the  secretary 
problem),  but  in  which  the  value  of  each  option  may  only  be  inferred  from  ex¬ 
periments.  Decisions  have  thus  to  be  made  ci>ncerning  both  the  acceptance 
and  rejection  of  each  option  and  the  degree  of  experimentation,  (ieneral  pro¬ 
perties  of  the  optimal  policy  are  derived,  and  an  algorithm  is  obtained  for  the 
solution  in  a  special  case.  This  special  case  suggests  a  heuristic  rule  for  more 
general  situations,  the  performance  of  which  rule  has  been  investigated  by  a 
Monte  Carlo  study. 


I.  INTRODUCTION 

We  consider  a  sequential  decision  problem  in  which  a  known  finite  number  of  options, 
having  values  drawn  from  a  known  probability  distribution,  are  presented  in  random  order  to 
the  decision  maker  who  cannot  observe  the  values  directly.  The  decision  maker  has  to  perform 
a  number  of  tests  on  each  option  (for  instance  by  some  form  of  sampling)  in  order  to  make 
inferences  about  its  unknown  value,  and  must  finally  decide  on  the  acceptance  or  rejection  of 
this  current  option  before  the  next  becomes  available.  We  assume  that  the  total  number  of 
tests  that  can  be  performed  is  strictly  bounded  above  (owing,  for  instance,  to  budgetary  con¬ 
straints).  According  to  a  criterion  to  be  discussed  in  the  next  section,  only  one  option  may  be 
chosen. 

The  problem  sketched  above  is  a  simplified  representation  of  a  number  of  praett:  :l  situa¬ 
tions.  A  mining  development  company  may,  for  example,  obtain  options  on  various  mineral 
deposits  from  time  to  time,  but  have  to  make  a  decision  on  each  (either  giving  up  the  option  to 
another  or  committing  its  resources  to  developing  a  mine  at  that  site)  before  prospecting  on  the 
next  site  can  even  start.  The  value  of  each  mineral  deposit  is,  of  course,  unknown  to  the  com¬ 
pany  and  must  be  estimated  by  bore-hole  samples.  These  are,  however,  expensive,  and  a  lim¬ 
ited  number  only  can  be  afforded  before  a  final  decision  has  to  be  made. 

A  similar  situation  may  be  faced  in  the  procuring  of  major  capital  equipment,  where 
expensive  operational  tests  may  be  necessary  to  select  between  alternatives,  and  where,  for 
practical  or  political  reasons,  negotiations  with  only  one  potential  supplier  at  a  time  is  possible. 
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These  practical  situations  are  generally  more  complex  than  the  decision  model  discussed 
here.  Nevertheless,  a  model  of  this  nature  can  be  used  to  gain  valuable  understanding  of  the 
real  problem,  as  discussed  in  some  detail  in  Stewart  {91. 

It  may  be  recognized  that  the  problem  described  in  the  opening  paragraph  is  essentially  a 
generalization  of  the  well-known  ‘secretary’  or  ‘dowry’  problem  in  the  case  of  values  drawn 
from  a  known  distribution  (for  instance,  the  ‘full  information  game’  of  Gilbert  and  Mosteller 
[31),  extended  to  allow  for  imperfect  information  (i.e.,  observation  errors)  on  the  value  of  each 
candidate.  Note  that  we  use  ‘information’  in  a  different  sense  from  that  of  Gilbert  and  Mos¬ 
teller,  where  ‘full  information’  refers  to  complete  knowledge  of  the  distribution  from  which  the 
values  are  drawn,  but  where  observation  of  each  value  is  implied  to  be  error-free.  The 
antithetical  case  of  those  writers  is  such  that  only  relative  ranking  of  candidates  is  possible,  but 
in  our  model  even  ranking  may  be  in  error  owing  to  observation  errors  (even  though  ‘full 
information’  concerning  the  underlying  distribution  is  assumed  to  be  available). 

MacQueen  [41  has  treated  a  somewhat  similar  problem,  involving  an  unbounded  number 
of  options  in  a  random  sequence,  but  with  a  cost  incurred  in  obtaining  each  new  option.  In  his 
problem  some  imperfect  information  is  available  on  receipt  of  each  option,  and  further  infor¬ 
mation  can  be  obtained,  at  a  cost,  by  executing  a  single  test  on  the  option.  The  present  prob¬ 
lem  differs  from  that  of  MacQueen  in  three  main  aspects: 

( 1 )  the  number  of  options  is  bounded; 

(2)  the  degree  of  testing  is  variable  (under  control  of  the  decision  maker),  and 

(3)  expenditure  on  testing  is  strictly  limited. 

The  p.v'Hiem  is  also  related  to  recent  work  on  partially  observable  Markov  processes  (see, 
for  instance,  Smallwood  and  Sondik  [71,  Sondik  [8l  and  White  [111).  In  our  case  we  have 
sufficient  special  structure  to  warrant  special  attention,  but  there  are  also  some  differences, 
especially  in  respect  to  the  variable  degree  of  testing  and  the  budgetary  constraints. 

In  Section  2  the  problem  is  formulated  more  precisely  and  the  decision  criterion  dis¬ 
cussed. 

General  properties  of  the  solution  under  certain  mild  assumptions  are  developed  in  Sec¬ 
tion  3.  In  particular  it  is  demonstrated  that  given  a  realization  of  a  sufficient  statistic  y  for  the 
value  of  the  option  based  on  the  tests  conducted,  there  exist  two  critical  values  u  and  w  such 
that  the  option  is  rejected  for  y  ^  u,  accepted  for  y  >  fv,  and  further  tested  otherwise.  These 
critical  values  depend  on  the  numbers  of  options  and  permitted  tests  remaining,  and  on  the 
number  of  tests  already  conducted  on  the  option. 

In  Section  4  a  special  case  is  introduced,  for  which  a  specific  algorithm  for  computing 
these  critical  values  can  be  obtained.  Not  only  is  this  algorithm  of  use  in  its  own  right,  but  the 
special  problem  suggests  an  adaptation  of  its  use  as  a  heuristic  for  more  general  problems. 

Section  5  consists  of  a  discussion  of  a  Monte  Carlo  investigation  of  the  behaviour  of  this 
heuristic. 
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2.  PROBLEM  FORMULATION  AND  DECISION  CRITERIA 

We  shall  throughout  the  paper  denote  random  variables  by  capital  letters,  and  their 
specific  realizations  by  the  corresponding  lower  case  letters.  Conditional  probabilities  of  the 
form  PrlZl  K  —  j'l  will  for  convenience  be  written  PrlZl^],  and  a  similar  convention  will  apply 
to  expectations,  probability  distributions  and  density  functions.  In  order  to  avoid  technical 
difficulties  we  shall  always  assume  that  the  probability  distributions  have  densities.  The  random 
values  Z„  and  observations  (defined  below)  will  be  assumed  to  be  real-valued  scalars. 

Now  assume  that  a  known  number  N  of  options  are  to  be  presented,  and  that  the  value  of 

option  «(/!  =  ! . N)  is  the  unobserved  random  variable  Z„,  drawn  from  a  known 

probability  distribution  F(.  ): 

Fiz)  -  PrlZ„  <  zl. 

Inferences  concerning  the  actual  realization  of  Z„  are  to  be  made  on  the  basis  of  observa¬ 
tions  on  r  independent  random  variables  Y„^,  y„2 .  y„r.  (The  choice  of  r  is  discussed 

later).  The  probability  distribution  of  conditional  on  Z„  =  z„  is  given  by  the  known  func¬ 
tion  GCvii^lz,,),  with  corresponding  density  /fCvi^lz,,).  Adopting  a  Bayesian  approach,  the  prior 

distribution  of  Z„  is  F(z„),  while  the  posterior  distribution  is  given  by  F„,(z„\y„\ . y„r)  for 

r  >  1; 

(I)  F,„(z„\y„^ . .v,„)  =  K  j?(.y,„|z)rff„,,_i  (zly„i . .v,,.,  ,) 

where  K  is  an  appropriate  integration  constant  and  where  for  convenience  F„^•,{z)  is  defined  as 
F(z). 


In  setting  up  decision  criteria  two  factors  have  to  be  considered,  viz.  the  cost  of  observa¬ 
tions  and  the  objective  of  selecting  a  satisfactory  option.  These  factors  could,  in  principle,  be 
handled  in  a  variety  of  ways,  but  for  practical  reasons  it  is  necessary  to  adopt  an  approach  that 
shows  promise  of  yielding  workable  solutions.  We  shall  thus  assume  the  following  structure  for 
the  problem: 

( 1 )  a  fixed  total  number  of  observations  R  will  be  allowed,  which  is  equivalent  to  assum¬ 
ing  a  fixed  budget; 

(2)  the  objective  of  the  decision-maker  is  to  maximize  the  expected  value  of  the  option 
chosen. 

Now  let: 

y,„,,(.y,i\ . v,„)  A  expected  optimal  value  of  the  option  chosen,  given  that  option  n  is  under 

consideration  Hhe  first  n  -  I  having  been  rejected),  that  r  observations  K„|  =  y„i . 

Kir  ”  y,u  have  been  made  on  «,  and  that  altogether  p  observations  have  been  made  on  the 
first  //options; 

and  then  let 

Kiifi^yii] . Jii.t  A  flyHi(,(.v„j . .v,j^)l.v„i . y,if  il. 
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In  evaluating  the  expectation  in  the  latter  definition  we  use  the  posterior  distribution  of 
K,„,  i.e.,  we  use  the  probability  density  for  Y„,  defined  by; 

(2)  g«.r-\(y\y„i . >’<..^-1)^  «0'l2)df„,,-l(zLv’„l . 3'n.r-l)- 

For  r  >  \,  p  <  R  and  n  <  N,  three  options  are  available  at  this  stage,  viz. 

(i)  10  accept  option  w, 

(ii)  to  make  another  observation  on  /r, 

(iii)  to  reject  option  w,  implying  that  option  n  +  1  and  the  first  observation  on  it  become 
available. 

We  thus  have  the  following  expression  fory^pO'ni . y„,): 

(3)  . y„r)  =  Max  (£{Zjy>„, . 

1 . 1  1 . 

where  we  note  that  F,,^,  ,  does  not  depend  on  any  observations.  Then  by  invoking  a 
dynamic  programming  argument  (see,  for  example,  Chow,  Robbins  and  Sigmund  (2),  Section 
3.2),  we  have  the  following  recursive  relationship  for  r  >  1,  p  <  /?  and  n  <  N: 

(4)  Krp{y«\ . y«.r-0  =“  iMax  {£(Zjy,, . y); 

•••.K,  i.t);  F„.,|  ,  ,(y|y„,,  . . .  y„,._|)  dy. 

For  r  =  1 ,  p  <  R  and  11  <  N  the  same  form  as  the  above  holds,  with  a  null  argument  list  for 
F„|p  and  using  implied  by  (2)  (from  £„,,(•)  =  £(•))  in  place  of  ^„,_|(  ).  When  w  =  M 
this  option  must  be  accepted,  and  the  expectation  of  its  expected  value  after  further  sampling  is 
the  current  expected  value.  When  p  =  /?,  either  the  current  option  or  the  (w  +  l)-th  (without 
observation)  has  to  be  accepted.  All  these  cases  can  be  accounted  for  by  the  relationship  (4)  if 
the  following  definitions  are  used  as  boundary  conditions; 

F,„ « + 1  =  for  r  >  1  and  all  h  <  /V 


K;i  zdF(z)  A  p,  for  all  n  <  A 


Fn  1,,  ■=  p 

F\>i.i,,  = 


for  all  p  <  £  +  1 
for  all  p. 


The  recursive  relationship  (4)  now  holds  for  1  ^  h  ^  /V,  I  ^  p  <  R,  1  ^  <  p  - 

and  also  defines  the  optimal  policy  which  is  to 

(i)  select  option  ff  if 

£(Z„l.v„i,  ....  .v.M )  ^  + 1  ./.+■  1  (Tki . and  F„.^i 


(ii)  take  another  observation  on  n  if 

FB,r  + l.p  + 1  (Tb I . Tnr)  ^  . Tiir^  and  F,^.|  j  p  +  i'. 
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(iii)  reject  option  »i  otherwise. 

Quite  clearly  any  general  solution  to  (4)  would  be  very  difficult,  if  not  impossible,  to 
obtain  in  practice.  Nevertheless,  we  obtain  in  the  next  section  some  general  solution  properties 
which  may  be  useful  in  specific  cases.  An  actually  soluble  problem  is  presented  in  Section  4, 
and  the  use  of  this  result  as  a  general  rule  is  discussed. 

3.  GENERAL  SGL'JTION  PROPERTIES 

1\vo  assumptions  regarding  the  distributions  f  and  0  will  be  invoked  in  this  section  at 
various  times.  These  assumptions  are  stated  first. 

ASSUMPTION  (Al ).  The  distributions  are  such  that 

(i)  given  observations  .v„i . .v,„,  there  exists  a  scalar-valued  sufficient  statistic  y,„  for 

(ii)  y„,  can  be  recursively  generated  (given  a  suitable  definition  of  y„o)  as  a  function  of 
y„  r- 1  and  y„r  which  is  nondecreasing  in  each  of  its  arguments; 

(iii)  E[Z„\y,„\  is  a  nondecreasing  function  of  .v,„; 

(i-.  the  posterior  probability,  Pr(  Y,„  ^  yl.v„,  i)  is  nondecreasing  in  ,v„,, .  |  for  all  .v. 

Assumptions  (i)  and  (ii)  together  imply  that  the  dynamic  programming  recursion  formu¬ 
lae  (4)  requires  only  .v,„  as  stale  variable,  rather  than  the  full  observation  vector.  Assumptions 

(iii)  and  (iv)  are  introduced  in  similar  spirit  to  assumption  2"  of  MacQueen  l4).  The  require¬ 

ment  is  that  the  probability  mass  of  the  observations  should  follow  the  true  underlying  value. 
More  specifically  (iii)  requires  that  posterior  inferences  on  z„  should  in  turn  follow  the  observa¬ 
tions,  while  conversely  (iv)  requires  that  posterior  probability  mass  on  future  observations 
should  follow  the  inferred  underlying  value  through  the  sufficient  statistic  Assumption 

(iv)  is,  in  a  sense,  analogous  to  the  'increasing  failure  rate’  assumption  of  reliability  theory.  In 
our  case,  however,  it  simply  implies  that  high  observation  values  should  not  decrease  the  poste¬ 
rior  probability  of  future  high  observation  values  (i.e.,  a  form  of  consistency  assumption). 

In  fact,  (Al)  may  be  expected  to  hold  quite  widely.  It  holds,  for  example,  when 
Z„  ~  A(0,  1)  and  Y,,,  ■—  N^z„,  o-’),  in  which  case: 

.V,„  =  7  L  Ym 

/-I 


or 


and 


y,„  -  {(r  -  -I  ^  y„Mr 
y„(i  -  0.  say  (arbitrarily) 


EzXy. 


-F  r 


(>•  -  . 
+  r  -  1 


+  r) 
a-^  +  r  —  1 
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The  normal  case  also  satisfies  the  second  assumption  below,  which  is  really  an  extension  of  (iii) 
and  (iv)  of  (Al),  now  requiring  a  certain  consistency  between  the  probability  mass  of  the 
observations  and  the  true  value  in  the  tails  of  the  distributions  as  well. 

ASSUMPTION  (A2);  (Al)  holds,  and  for  any  «  >  0  there  exist  >'|  and  such  that 
1 )/»  <  .fil  and  {  Y,„  >  w-il  have  nonzero  probability  and  for  any  m  >  n 

(i)  y,,,  <  .Vi  implies 

PrIZ„  >  z„  lT,,l  F,Jz\y,JdF(z)  >  1  -  € 

(ii)  y,,,  >  .v":  implies 

PrlZ,„  >  cJ.Vml  <  €. 

We  now  consider  certain  general  properties  of  the  solution.  First,  however,  we  prove  the  fol¬ 
lowing  lemma. 

LEMMA;  If  (Al)  holds,  then  F„,,,(J'„., . |)  (=  ^,„,,(y,i\ . see  comment  after 

assumption  (AD)  is  a  nondecreasing  function  of  y„,  i,  which  is,  furthermore,  such  that: 

D,„,,(y,,,  |)  A  l£'Z„l.v,,,  ,1-  y,„,.(y,,,  |) 

is  a  nonposilive,  nondecreasing  function  of  v,,.,  i-  (Note  that  by  virtue  of  (Al)  (i),  conditional 
expectations  depend  only  on  .v„,  |.) 

PROOF.  By  assumptions  (Al)  (ii)  and  (Al)  (iii)  it  follows  that  for  given  y,„  =  y. 

E[Z„\y,„  I,  y,„l  =  £(Z„|.v,„(.v,,,  |.  y)l 
is  a  nondecreasing  function  of  .v„  .  i  and  y. 

The  boundary  conditions  on  y,,,,,  imply  that  for  any  r(l  ^  r  <  R)  and  given  y„,  =  .v: 

■xnikI.cm . .v\<)  “  y\fK^y\.i  !•  y^ 

=  £(Z^(,V\  ,  1.  >-! 

while  for  //  <  A 

y,„i<^y,.,  I- >’)  =  Maxl£(zjT,„(.v,,,  I.  >)hM)- 

(Here  and  elsewhere  in  the  proof  'y,„/((F„,  |,  y)  is  a  compressed  form  of  y,„K(.V',„(J'„,,  i.  y)) 
and  similarly  for  the  other  functions.) 

Thus,  for  all  n  ^  A  and  r  ^  R  and  a  given  value  for  y,  y„rK  (Tn,r-i..v)  is  non-decreasing 
in  y„,-i  and  y.  Now  define  y„.r~i  and  y.  Now  define 

^,„f<^y,u  I  -  .t  )  ^  E{Zjy„,  I.  v)  -  y,„/<^y,u  i.  .v) 

=  MinlO;  £{Z„l.v,„(.v„,,  .v))  -  /al. 

Clearly,  8,„/,(.v„,  |.  v)  is  also  nondecreasing  in  .v„,  i  and  y,  and  is  nonpositive. 

Now  define  the  following  probability  density  function  on  y. 
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with  corresponding  distribution  function  G„r  ((.v).  This  distribution  is  defined  to  be  indepen¬ 
dent  of  any  perturbation  of  .p,, ,  i  around  the  nominal  value. 


Since  y(-)  and  6(  )  are  nondecreasing  in  .v«./  i  for  every  y,  it  follows  that  their  expecta¬ 
tions  with  respect  to  the  distribution  defined  by  g„,  (I)  are  also  nondecreasing  in  r  |.  Thus, 
defining  for  any  J'  (not  necessarily  the  nominal  value  y,,.,  - 1): 


=  •'K 

I.  y)  i^y^dy 

and 

b„rR  (y'li.r 

,.y)ft,,,  i(y)a!v 

we  have  for  any  h  >  0: 

1  +(i)  >  K>R(y>u 

“  ^,„R^y,u 

and 

b,„R 

1  +/))  >  b,„n(y.,,, 

1*  “  X,!  ' 

1.  y>g,,,  i(y!.p, 

—  £lZ„ly„,  |)  —  i) 


i)dy 


Now  consider  the  expectations  of  ■)  and  )  when  y  has  distribution  function 

G„ 1  (,v ly„ 1  +h).  By  assumption  (Al)  (iv),  if  this  distribution  differs  from  G„r_|(y),  the 
difference  is  a  shift  in  probability  weight  to  higher  values  of  y,  and  thus,  since  )  and 

.  ■)  are  nondecreasing  in  y.  we  have  for  h  >  0: 

/  +>l)  >  f',,«(y,,r-l  +/>)  >  KrR^yn.r-O 

D„rR(y„.r-t  +/l)  >  D„ft(y,,-,). 

Furiherrdore,  taking  expectations,  it  is  also  clear  that  D, „/({}•„,  i)  inherits  the  nonpositivity  of 
S,„K(y„  r  .M.  y).  The  lemma  thus  holds  for  p  =  /?.  For  other  values  of  p,  the  )  values  can 
be  obtaified  by  backward  induction  from  the  T,„«(  )  computed  above  and  from  the  boundary 
condition. 


Fm,,  =  p,  p  ^  R  +  \  . 

We  thus  assume  that  the  lemma  holds  for  n.r -f  1,  p-Fl,  i.e..  that  F„,^i.,,+i(y,„)  and 
are  nondecreasing  functions  of  y,„  and  that: 

^11.  r  *  I  .  I  (.V,;/  1  ^  0  . 

Subtracting  the  expression  for  y„,,,{y„,  i,  y)  given  by  Equation  (3)  (where,  as  above, 
y„i . y„  ,  1  is  replaced  by  y„,  i  and  y,„  =  y)  from  £  lZ„|y,„(y„,  y)i,  we  have 

S,„,,(.v„,,  y)  =  Min{0;£|Zjy„,(y„,,  y))  -  F„,^i.,,.nly,„(y„,,.- 1,  y)l; 

£'{Zj.v,„(y„.,  I.  y)}  -  F„>i  i,„..il. 

=  Min  10;  £>„,  +  , {.v„.(y„,,  y)); 

fUJy^rCvn.r-i.  y))  - 

Since  F„+i  |  is  a  constant  not  dependent  on  y„,  j  or  y  we  have  from  (3)  and  the  above,  and 
using  (Al)  (ii)  and  (Ai)  (iii),  that  y,„,.(y„,r  i.  y)  and  8,„,,(y„,  |,  y)  are  nondecreasing  func¬ 
tions  of  y„ ,  I  and  y,  and  that  8„,^,(y„.,  |.  y)  is  nonpositive.  In  the  same  way  as  above  it  can  be 
shown  that  F,„,.(y„,,  |)  and  £>„r,,(.F„.,  |)  are  nondecreasing  iny„,  j  and  that 
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D„rph>n.r-\)  <  0-  II 

The  lemma  follows  by  induction. 

Lemma  1  may  be  seen  to  be  a  special  case  of  a  wider  class  of  results  derived  by  Topkis 
[10],  where  conditions  under  which  optimization  problems  yield  solutions  which  are  isotonic  in 
a  parameter  are  discussed.  In  the  present  case,  however,  not  only  is  the  direct  proof  relatively 
simple,  but  it  directly  yields  the  structure  of  the  optimal  policy.  In  fact,  by  the  optimality  prin¬ 
ciple  of  the  last  section,  there  is  implicit  in  the  proof  also  the  demonstration  of  the  following 
theorem. 

THEOREM  1;  If  (Al)  holds,  then  the  optimal  policy  is  defined  by  numbers  and 
for  all  r,  p  such  that 

(they  may  be  equal),  and 

(i)  option  n  is  rejected  if  y,„  < 

(ii)  option  n  is  accepted  if  y„,  > 

(iii)  a  further  sample  is  taken  otherwise. 

This  is  a  result  which  is  intuitively  fairly  obvious  although  difficult  to  prove,  and  it  is  simi¬ 
lar  in  spirit  to  related  results  of  Rosenfield  l6l  and  MacQueen  [4].  General  conditions  for  the 
optimality  of  structured  classes  of  policies,  such  as  that  of  Theorem  f,  have  been  derived  by 
Porteus  (5|.  Thus,  as  for  Lemma  I,  Theorem  I,  is  a  special  case  of  a  more  general  result.  Our 
derivation  has,  however,  allowed  the  specific  form  of  structuring  to  be  developed  construc¬ 
tively,  and  thus  Theorem  1  is  in  a  form  suitable  for  direct  practical  implementation. 

This  structure  of  the  optimal  policy  is  important  for  a  number  of  reasons.  Certainly  it  is 
easier  to  store  w,„„  and  m,„„  values  than  to  store  y, „,,{■)  for  all  values  of  its  argument.  In  some 
cases  it  may  even  be  possible  to  compute  and  directly,  although  we  have  not  found 
such  an  example.  Furthermore,  once  this  structure  of  the  optimal  policy  is  known,  various 
heuristics. can  be  proposed  for  the  problem,  based  on  assumed  or  heuristically  derived  values 
for  the  M,„,,  and  w,,,,,. 

We  nov.  state  and  prove  a  further  property  of  the  optimal  policy  under  the  stronger 
assumption  (A2).  This  result  is  given  here  not  only  because  it  is  an  aid  to  intuitive  under¬ 
standing  of  the  problem,  but  also  because  it  may  suggest  heuristic  derivations  for  the  m,,,,,  and 
w,„,,  above,  based  on  approximations  for  ,). 

THEOREM  2;  If  (A2)  holds  and  y,„  €  Ij,.  .v„l  for  all  n  and  r  (we  may  have  >'/  •=  -«» 
and/or  .v„  =  «>),  then 


lim  y„„,(y,,r  I 

.. .  it.', 


^  “  ^H+l.  I./.+  I 


lim  y„r,,(y,,,  |)  =  f  \.E\Z„\y,^{y„,  ,,  v„,))|.y„,,  ,1. 

(Note  that,  although  the  second  expression  above  is  in  a  rather  cumbersome  form,  it  is  in  most 
practical  applications  quite  easily  evaluated). 
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PROOF:  By  assumption,  for  small  enough  y„,  i  it  will  be  optimal  to  reject  option  n 
immediately,  giving; 

ywpCVn.r— I*  .V j  ^«  +  LI./i  +  I 

for  all  .v„ ,  I  and  and  the  first  limit  holds. 

Similarly,  for  large  enough  .y,,^  j,  option  n  is  immediately  accepted,  giving 
"y - !» 

and  the  second  limit  follows.  II 

In  this  paper  we  have  not  attempted  to  develop  any  heuristics  on  the  basis  suggested 
above.  Rather,  we  introduce  in  the  next  section  a  particular  case  in  which  the  exact  solution 
can  be  computed.  Not  only  is  this  case  of  some  interest  in  its  own  right,  but  it  suggests 
another  heuristic  approach  for  the  general  case.  The  results  of  a  Monte  Carlo  study  of  the  per¬ 
formance  of  this  heuristic  are  presented  in  Section  5. 

4.  BETA-DISTRIBLTED  VALUES  WITH  BINOMIAL  TRIALS 

In  this  section  we  consider  the  case  in  which  the  Z„  are  independently  distributed  accord¬ 
ing  to  a  Beta  distribution,  with  parameters  o  and  j3,  i.e.,  Z„  €  (0,  U  and  Fi  )  has  density: 

'  r(«)r(/3)  ‘  ■ 

The  observations  Y,„  are  of  a  simple  binomial  success/failure  form,  with  success  probabil¬ 
ity  r„.  This  is  a  somewhat  simplified  model  of  many  real  situations  in  which  r„  represents,  for 
each  option,  a  certain  degree  of  satisfaction,  and  the  .v,,^  represent  either  a  simple  sampling 
(e.g.,  defectives  in  a  batch)  or  the  results  of  expert  opinions  (each  declaring  the  option  ‘satis¬ 
factory’  where  r„  is  now  the  fraction  of  all  experts  who  would  find  the  option  satisfactory). 

Since  y,,,  €  {0.  I},  we  can  define 


.V„(i  =  « 


«  +  £  y»i 


y„.r  I  +  y„r- 


Then,  a  posteriori,  the  uncertainty  in  z„  can  also  be  represented  by  a  Beta  distribution  with 
parameters  y,„  and  (a  -E  /3  r  -  J,„),  i.e., 

E  Zjy,„  -  yj(a  +  (3  +  r)  -  Pr(  -  1 

Using  the  same  boundary  conditions  as  in  Section  2,  we  thus  have  for  n  <  N,  r  <  p,  p  <  R-. 

(5)  ynr,Cyn.r-0  »  ' i  ^aX  -El);  F,.,.,.p  +  i| 

a+P+r-\~y„,.,  j  K.r-t  ^  i  „  1 

a-E)3  +  r-l  L '^'.r+i.P+tO',.r-i),  (',+l.i.p+ij 


\  a  ^~r  '  ^'.+..Lp+i 


Ki.r+l.p+tO'p.r-l)*  Ki+l.l.p+1 
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Now  because  the  state  space  of  y„,  is  finite,  i.e.,  €  (0.  1,  it  is  in  principle  possible  to 

compute  V,„i,(y„  ,  . ,)  for  all  values  of  n,  r,  p,  andj,,.,  i.  This  would  be  achieved  by  starting  for 
each  n  <  N  with  each  value  of  s  in  I  <  s  <  /?  —  /i,  and  then  computing  by  a  backward  recur¬ 
sion  for  y  =  0.  I .  (s  -  I),  (y„,  |)  where  r  =  s-j  and  p-R~j  for  all 

.V,,.,  1  €  lO,  1 . r  -  1).  Note  that  in  doing  so  we  may  discard  all  +  values  after 

computing  the  )  values  and  storing  the  u„,,,  and  h,,,,,  values.  Apart  from  this  the  only 
other  intermediate  results  requiring  storage  are  the  (R  -  n)  values  of  Kz+i  i.i.+  i  (p  ■=  w  +  2, 

n  +  3 . R  +  I).  Thus,  at  most,  HR  -  I)  intermediate  storage  positions  are  necessary. 

Finally,  the  policy  variables  w,„^,  and  must  be  stored,  but  this  need  not  be  in-core;  approxi¬ 
mately  2NR-  values  would  be  generated. 


In  order  to  achieve  this  computation,  the  following  number  of  evaluations  of  (5)  are 
required; 

\  \  K  t  \  II  ^  ^  "  c  ( c  4-  n 

I  I  1(^-2)  =  I  I 

»/■»  I  S-  1  /"•()  ft""  I  s- 1  ^ 


1  '  ' 

I 


{R-  n){R-  n-^l){2R-2n  +  \)  ^  <.R  -  n)  {R  -  n  +  ]) 


,i-\ 

s  I 


-|y  £  [2/?  (/?-»- n  (/? -t-2)  -  2(3/?- -l-6/J-t-2)«-l-6(/f  4-  I)/j--2/;'). 


-I- 1 


/?(/?  -h  l)(/?-l-2)(/V-l)  _  OR-+bR+2)  iN  -  ])N 
12  12 

(R  +  \){N~l)N(2N-\)  _  \HN-  1)^ 

12  24 


For  moderate  values  of  N  and  R  (e.g.,  10  and  10%  respectively,)  this  computation  is  indeed 
quite  feasible,  since  evaluation  of  (5)  is  extremely  simple  (three  additions/subtractions,  six 
multiplications/divisions  and  four  comparisons). 


Although  the  solution  is  not  very  elegant,  its  feasibility  is  of  interest.  In  particular,  we 
consider  the  case  in  which  a  =  ^  =  1,  i.e.,  the  case  of  uniform  distribution  on  [0,  1).  For  a 
known  distribution  function  C(  ),  Z„  can  always  be  transformed  into  a  variable  Z,',  which  is  uni¬ 
formly  distributed.  Maximizing  the  expected  value  of  the  Z,',  chosen  is  equivalent  to  maximiz¬ 
ing  the  fractile  value  of  Z„'  and  thus  of  Z„  (which  also  minimizes  the  probability  that  one  given 
later  option  will  improve  upon  option  «).  This  is  in  fact  a  quite  satisfying  objective  in  the  gen¬ 
eral  case  (cf.  the  ‘uniform  game,’  paragraph  5a  of  Gilbert  and  Mosteller  (31). 

Now  if  the  Yh,  represent,  as  above,  binomial  trials  on  the  uniformly  distributed  Z^J,  then 
any  event  of  the  form  I f  1  is  equivalent  to  the  event  { [estimated  probahMity  that 
Z,„  <  r„,  for  any  given  ni  >  «]  <  f/(r  +  2)}. 

For  general  observations  y„,  on  z„  this  probability  can  also  be  computed:  it  is  in  fact  given 
by  where 

(6)  7),,,  -  ll  -  F„(2lv,„)lc/f(r). 

(It  is  easily  confirmed  that  in  the  uniform/binomial  case  -  .v,„/(r  +  2).)  This  suggests  that  a 
heuristic  policy  may  be  defined  by  the  following:  where  and  w,',,,  are  the  critical  values 
computed  in  the  binomial  case  and  with  a  “  j3  -  1 , 
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(i)  option  n  is  rejected  if  4  u',„,J(r  +  2); 

(ii)  option  n  is  accepted  if  +  2); 

(iii)  a  further  sample  is  taken  otherwise. 


S.  MONTE  CARLO  RESULTS 


In  order  to  test  the  heuristic  suggested  at  the  end  of  the  last  section,  a  series  of  Monte 
Carlo  studies  were  performed.  The  example  selected  for  these  studies  was  chosen  more  to 
demonstrate  performance  under  fairly  extreme  conditions  than  to  represent  any  particular  prac¬ 
tical  situation.  We  give  hrst  a  description  of  this  example. 


The  values  Z„  are  drawn  from  an  inverse  gamma  distribution  with  shape  parameter  3  and 
scale  parameter  I,  i.e.,  if  we  define  X„  =  l/Z„,  then  the  distribution  of  X„  has  density  ^(x): 

0(x)  =  y  X- c  '. 

The  observations  T,„  are  drawn  from  the  exponential  distribution  with  mean  Z„,  i.e., 

^Cvlz,)-  kxp(-y/z„)]/z„. 

A  sufficient  statistic  for  z„  is  thus 


y,„  = 


/-I 


It  is  not  difficult  to  show  that  (Al)  (in  fact  (A2))  holds.  The  value  of  tj,,,  of  the  previous  sec¬ 
tion  is  easily  obtained,  although  in  a  form  slightly  different  from  that  of  Equation  (6).  We  note 
first  that  the  posterior  distribution  of  x„  (  note,  not  z„)  is  also  of  the  gamma  form  with  density 
defined  as  <<),„  (x)  given  by; 

(J',,,+l)'‘^'  expl-(J,„+l)x] 


<f»,„(x)  = 


r(3-E  r) 


The  probability  that  Z,„  <  z,,  for  any  given  m  >  n,  \s  equal  to  the  corresponding  probability 
that  X„,  >  x„.  Conditional  on  x„  (  i.e.,  on  z„)  this  probability  is  given  by 


Prl3r,„  >  =  x„l  =  \x'e  'dx 


1  -t-  X„  -E  y  X,f 


exp(-x„). 


Taking  the  expectation  of  this  conditional  probability  with  respect  to  the  posterior  distri¬ 
bution  for  X„  conditional  on  y,„,  we  obtain  the  required  expression  for  tj,,,  as  follows: 


Viif 


f"  (>'„,-H)''^'exp[-(>'„,-E2)xl 
J"  1(3  r) 


dx 


y„r  +  2 

3  +  r 
y,„  +2 


(3  -E  r)(4-Er) 
+2)' 


W  ith  these  results  the  proposed  heuristic  can  be  applied. 
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In  ihe  studies  performed,  R  =  \0N  throughout.  Now  it  is  important  to  note  that  simply 
to  take  10  observations  on  each  value,  and  to  use  the  mean  of  these  as  the  true  value  in  one  of 
standard  models  (as,  for  instance,  in  Gilbert  and  Mosteller  [3])  can  lead  to  substantial  errors. 
For  example,  around  the  mean  value  of  X„  (i.e.,  3)  the  variance  of  the  estimate  of  z„  is 
0.03333,  and  the  one  standard  deviation  limits  of  this  estimate  is  [0.1507;  0.S1S9I.  In  fact,  this 
range  within  which  the  estimate  is  most  likely  to  fall,  covers  approximately  65%  of  the  probabil¬ 
ity  mass  of  Z„.  Thus,  even  if  all  N  options  could  be  seen  simultaneously,  in  ordering  the 
options  quite  considerable  errors  could  occur. 

The  Monte  Carlo  studies  with  the  heuristic  rule  were  carried  out  for  four  values  of  /V,  viz. 
4,  6,  8  and  10.  In  each  case  1,000  sets  of  A  values  were  generated  by  a  standard  pseudo¬ 
random  number  generator.  The  cutoff  points  and  had  been  computed  previously  and 
stored.  For  each  set  the  rule  was  applied,  and  the  rank  and  value  of  z,,  for  the  option  chosen 
was  recorded.  In  this  way  the  following  comparisons  could  be  made  with  optimal  performance 
in  perfect  information  models: 

(a)  Mean  fractile  of  the  option  selected  (i.e.,  the  expected  probability  that  the  chosen 
option  exceeds  a  random  draw  of  Z„,;  if  Z„  is  transformed  to  a  random  variable,  U„ 
say,  which  is  uniformly  distributed  on  (0.  1]  then  this  corresponds  to  £t/„  for  the 
option  selected).  The  observed  mean  can  be  compared  with  the  corresponding 
optimal  expected  value  without  observation  error  ([31,  Table  II).  It  may  be  recalled 
that  maximization  of  this  value  is  the  objective  invoked  as  a  basis  for  the  heuristic. 

(b)  Frequency  of  selection  of  the  best  option.  This  can  be  compared  with  corresponding 
optimal  probabilities  in  the  standard  cases  with  no  observation  error  when  the  under¬ 
lying  distribution  is  either  known  ((31,  Equation  (3c- 1)  and  Tables  7  and  8)  or  unk¬ 
nown  (and  ranking  only  is  possible;  (31  Table  2). 

(c)  Mean  rank  of  the  option  chosen  (where  1  =  best).  This  can  be  compared  with  the 
corresponding  optimal  expected  rank,  if  ranking  only  is  allowed  (111.  Equations  (6) 
and  (7)). 

These  comparisons  are  shown  in  Table  1 

Clearly  the  heuristic  rule  compares  well  with  optimal  rules  when  there  are  no  observation 
errors.  In  particular,  the  rule  performs  substantially  better  than  the  optimal  rules  based  on 
error-free  ranking  of  the  options  only.  Of  more  importance,  however,  is  the  fact  that  perfor¬ 
mance  compares  favourably  with  optimal  rules  based  on  full  information  on  the  underlying  dis¬ 
tribution  and  perfect  information  on  the  values  in  hand.  This  advantage  is  particularly  evident 
for  the  mean  fractile  of  the  option  chosen,  the  basic  objective  of  the  heuristic. 

6.  CONCLUDING  COMMENTS 

We  have  introduced  a  class  of  sequential  decision  problems  which  generalize  the  "secre¬ 
tary"  or  "dowry"  problems.  Apart  from  general  solution  properties,  an  exact  solution  has  been 
found  in  principle  for  a  particular  case.  What  is  more  important,  it  has  been  shown  that  the 
solution  in  this  case  can  form  the  basis  of  heuristic  rule  for  more  general  situations,  which  rule 
has  been  validated  by  means  of  a  Monte  Carlo  study.  Admittedly,  the  rule  is  computationally 
convenient  only  for  rather  small  numbers  of  options,  but  it  must  be  noted  that  the  motivating 
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TABLli  1  —  Cumparimn  oj  Performance  oj  the  HeurisHc  Rule  and  Optimal 
Performance  with  Error-Free  Information 


Number  of  options  (N) 

4 

6 

8 

10 

IVIVUII  IICIVIIIV  VJI  IIIW 

selected. 

estimate,  case  H 
(standard  deviation  of  estimate) 
optimal  expected  value, 
case  0, 

0,727 

(0,007) 

0,742 

0,786 

(0,006) 

0,800 

0,818 

(0,006) 

0,836 

0,853 

(0,005) 

0,861 

Probability  of  selecting 
the  best  option: 

observed  frequency,  case  11 

0,607 

0,580 

0,516 

0,526 

(estimated  standard  deviation  of 
frequency) 

(0,015) 

(0,016) 

(0,016) 

(0,016) 

optimal  probability,  case  Oj 

0.655 

0,629 

0,616 

0,609 

optimal  probability,  case  O, 

0,458 

0,428 

0,410 

0,399 

Mean  rank  of  option 
selected: 

estimate,  case  H 

1,587 

1,734 

1,921 

1,989 

(standard  devialioi  of  estimate) 

(0,027) 

(0,035) 

(0,043) 

(0,049) 

optimal  expected  rank, 
case  0, 

1,875 

2,217 

2,400 

2,558 

(  ,isc  11  Itcui'isiic  rule  of  Section  4.  esiiniaics  baseil  on  the  Monte  Carlo  study 
(  ase  O,  oplintal  rule  given  iKrl'eci  infornialion  on  values  drawn  front  a  known  distrihution 
(  ase  < )  optimal  rule  given  perfect  information  only  on  tanks  of  the  observed  options 


examples  in  the  iniroducfion  (in  which  costly  examination  of  each  option  is  required)  will  gen¬ 
erally  involve  rather  few  options.  In  these  cases  the  model  and  solution  would  appear  to  have 
some  applicability 
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ABSTRACT 

In  this  paper  we  analyze  optimal  search  strategies  in  an  environment  in 
which  multiple,  independent  targets  arrive  and  depart  at  random.  The  analysis 
revolves  around  a  continuous  lime  differential  equation  model  which  captures 
the  time  dependent  nature  of  the  search  process.  We  explore  the  impact  on 
optimal  strategies  of  nonzero  travel  times  between  regions  as  well  as  differing 
target  arrival  rates.  We  derive  simple  closed  form  expressions  for  determining 
if  only  one  region  should  be  searched. 


INTRODUCTION 

In  this  paper  we  present  a  differential  equation  model  used  to  explore  optimal  search  stra¬ 
tegies  in  an  environment  which  involves; 

1 .  multiple  independent  targets 

2.  random  arrival  of  targets 

3.  random  departure  of  targets 

4.  different  target  types  (mean  duration) 

5.  time  lost  in  travel  between  noncontiguous  regions. 

A  wide  range  of  target  types  exhibit  characteristics  similar  to  some  or  all  of  the  above.  This 
research  evolved  initially  from  research  into  crimes  but  has  application  to  the  following  target 
types,  a)  smugglers,  b)  terrorists,  c)  transiting  submarines,  d)  infiltrating  enemy  forces  and  e) 
groups  of  attacking  aircraft. 

After  a  brief  literature  review  (see  [13],  115],  and  117)  for  more  extensive  reviews),  we 
introduce  the  basic  differential  equation  model  and  apply  it  to  a  series  of  increasingly  more 
complex  problems.  The  analysis  considers  only  cyclical  search  strategies  and  focuses  on  the  two 
region  problem.  Our  goals  are:  (1)  to  determine  the  optimal  cyclical  strategy,  and  (2)  to 
develop  simple  analytic  expressions  for  when  a  region  should  be  excluded  from  search. 
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The  problems  we  address  are  as  follows; 

(1)  Two  regions  with  equal  target  arrival  rates,  no  time  lost  in  travel  between  regions. 

We  determine  for  this  problem  the  optimal  cycle  length  and  the  magnitude  of  the  impact  of 
using  optimal  search  strategies. 

(2)  Two  regions  with  equal  target  arrival  rales,  time  lost  in  travel  between  regions. 

For  this  problem  we  again  determine  the  optimal  cycle  length  and  also  develop  an  analytic 
expression  for  when  to  search  only  one  region.  This  expression  is  a  function  of  the  travel  time, 
and  the  departure  and  detection  rates  of  the  targets,  but  not  the  arrival  rate. 

(3)  Two  regions  with  unequal  target  arrival  rates,  no  time  lost  in  travel  between  regions. 

We  determine  optimal  cyclical  policies  and  an  analytic  expression  for  when  to  search  only  the 
high  target  region  which  is  a  function  of  only  the  ratio  of  the  departure  to  the  detection  rate. 
In  addition,  we  explore  the  impact  of  constraining  the  minimum  time  to  search  a  region. 

In  all  of  the  above  examples  we  consider  only  a  single  target  type  and  only  two  regions. 
We  close  with  a  brief  discussion  of: 

(4)  Two  regions  with  multiple  target  types. 

(5)  More  than  two  regions. 

Throughout  the  paper  we  evaluate  the  limiting  properties  of  many  of  the  equations  and, 
for  example,  study  the  impact  of  infinitesimally  short  cycle  times.  Although  these  cycle  lengths 
are  not  achievable  in  the  real  world,  the  results  provide  important  bounds  on  the  difference 
between  optimal  and  nonoptimal  strategies. 

1.  LITERATURE  REVIEW 

The  pioneering  work  by  Koopman  I9j,  [10],  and  lllj,  along  with  the  work  in  the  discrete 
analog  by  Charnes  and  Cooper  [3],  has  laid  the  foundation  for  the  continuing  expansion  of 
search  theory.  The  later  work  which  is  most  directly  applicable  to  our  area  are  those  papers 
which  explore  the  sequencing  of  search  effort.  Blackman  and  Proschan  [2]  analyzed  the  optimal 
sequencing  of  search  effort  among  a  series  of  regions  into  which  targets  arrive  randomly.  Gil¬ 
bert  [6],  and  later,  Kisi  [8]  analyzed  a  two-box  search  problem  with  the  objective  to  minimize 
the  expected  time  until  detection.  However,  their  work  is  not  extendable  to  situations  with 
departing  targets  l4].  Moore  [14]  analyzed  the  search  for  one  randomly  arriving  and  departing 
target  in  n  regions  solely  from  the  perspective  of  how  much  search  effort  to  allocate  to  each 
region,  but  not  the  sequencing  problem.  The  paper  closest  in  orientation  to  this  work  is  that  of 
Barnett  [1].  In  his  problem,  targets  arrive  in  a  Poisson  process  and  survive  for  a  random  dura¬ 
tion.  Barnett,  in  considering  a  discretized  search,  (1)  proves  optimal  strategies  are  cyclical,  a 
fact  we  use  in  applying  the  differential  equation  model,  (2)  finds,  analytically,  the  optimal 
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sequence  of  search,  and  (3)  generates  a  sufficient  condition  for  excluding  regions  from  the 
optimal  search  strategy.  We  develop  analogous  results  while  introducing  a  more  flexible  con¬ 
tinuous  search  problem  which  allows  for  travel  time  between  regions  and  the  possibility  of 
searching  a  region  and  not  discovering  a  target  that  is  actually  present. 

2.  DIFFERENTIAL  EQUATION  MODEL 

The  model  consists  of  two  differential  equations  for  each  region,  one  to  describe  the  sys¬ 
tem  when  no  search  is  in  progress  in  that  region,  the  other  to  describe  the  system  when  a 
search  is  in  progress. 

Let  S  -  the  state  of  the  system  (expected  value  of  the  number  of  targets  present). 

A  =  the  rate  at  which  targets  arrive  (assumed  independent  of  the  state  of  the  system). 

Targets  depart  from  the  system  in  either  one  of  two  ways.  Either  targets  leave  because  they 
have  finished  their  task  or  because  they  have  been  intercepted  during  a  period  of  search. 


F  =  the  constant  of  proportionality  for  the  rate  at  which  targets  finish  their  tasks.  (This 
will  depend  upon  the  duration  of  their  task.) 

/  =  the  constant  of  proportionality  for  the  target  interception  rate.  (This  will  depend  on 
the  size  of  the  region  and  the  observability  of  the  target.) 

Using  the  above  parameters  (S,  A,  F,  /),  we  can  write  the  following  equations  to 
describe  the  changing  system.  During  a  period  of  no  search,  the  system  is  described  by 

(1)  A  -  FS. 
dt 

The  equation  states  that  the  system  is  changing  because  targets  are  arriving  at  a  rate  A  and  are 
departing,  as  a  result  of  finishing,  at  a  rate  FS,  proportional  to  the  state  of  the  system.  The 
solution  of  the  differential  equation  is 

(2)  S  =  C,  exp(-ff)  +  AIF. 

/4/f  represents  the  steady  state  number  of  targets  if  no  search  were  carried  out. 

During  a  period  of  search,  the  equation  becomes 

(3)  ^~A-(F  +  I)S 
dt 

whose  solution  is 

(4)  S  -  CiCxpi-iF  +  Dt)  +  A/(F  A- I). 

AliF  +  /)  is  the  steady  state  number  of  targets  during  an  unending  period  of  search. 

These  equations  assume  that  the  arrival  process  of  targets  is  Poisson  and  that  the  duration 
of  a  target's  presence  is  exponentially  distributed.  S  is  therefore  the  expected  value  of  the 
number  of  targets.  This  can  be  shown  by  setting  up  queueing  type  equations  for  the  probabili¬ 
ties  P„,  where  n  is  the  number  of  targets  present  at  any  one  time. 
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3.  TWO  REGIONS-EQUAL  TARGET  ARRIVAL  RATES-NO 
TIME  LOST  IN  TRANSFER 

The  first  problem  to  which  we  apply  the  differential  equation  involves  two  identical 
regions.  For  now  we  will  focus  on  the  optimal  cycle  length  and  consider  only  equal  allocations 
of  search  effort  to  the  two  regions;  later,  we  will  show  that  in  fact  the  global  optimal  strategy  is 
symmetric.  In  addition,  we  will  consider  only  simple  cyclical  strategies  of  the  form  X  minutes 
of  search  in  /{ 1,  followed  by  X  minutes  in  R2  and  then  back  to  R1  for  minutes,  and  so  on. 
This  second  assumption  is  motivated  by  a  number  of  earlier  search  theory  papers  (1,  6,  8]  in 
which  the  optimal  search  strategies  were  cyclical. 

Our  objective  will  be  to  minimize  the  total  average  expected  number  of  targets  in  the 
combined  regions.  However,  because  the  regions  generate  independent  target  arrivals  and 
departures,  they  will  be  treated  separately  as  experiencing  alternating  periods  of  search  and  no 
search.  In  addressing  this  problem  we  will,  in  addition,  answer  simultaneously  the  following 
question: 

Given  that  a  searcher  is  unavailable  50%  of  the  time,  what  is  the  distinction  between  short 
numerous  intervals  of  search  and  a  few  long  intervals  of  search? 

The  three  steps  common  to  this  and  the  later  examples  are  outlined  as  follows: 

1.  Two  differential  equations  are  defined  for  each  region.  One  to  describe  the  dynamics 
of  the  region  while  it  is  being  searched,  the  other  to  describe  the  region  while  no 
search  is  going  on. 

2.  For  each  region,  two  boundary  conditions  of  the  following  form  are  established.  The 
target  level  in  each  region  at  the  end  of  a  period  of  search  must  equal  the  level  at  the 
beginning  of  a  period  of  no  search  and  vice  versa.  These  conditions  are  simply  a 
continuity  constraint  on  S,  the  number  of  targets  present,  not  allowing  discrete  shifts 
in  the  values  of  S  as  a  result  of  the  searcher  entering  or  departing  a  region.  Once  the 
boundary  conditions  are  defined,  each  of  the  constants  in  the  differential  equations  is 
determined. 

3.  The  average  target  level,  S,  in  the  total  area  is  calculated  by  integrating  each  of  the 
four  equations  over  its  respective  period  of  search  or  no  search,  summing  the  four 
values  and  dividing  by  the  cycle  length.  The  resultant  expression  for  S  is  a  function 
of  the  parameters  of  interest  in  each  of  the  particular  examples  and  is  subsequently 
analyzed  to  determine  an  optimal  strategy. 

In  both  regions,  /?i  and  /?2,  the  two  equations  describing  the  periods  of  search  and  no 
search  are  just  the  previously  defined  Equations  (1)  and  (3)  and  whose  solution  are  Equations 
(2)  and  (4). 

The  continuity  constrainton  S  is  used  to  generate  the  following  equation; 

(5)  Cj  exp  (-(f  +  /)X)  +  y4/(F  + /)  ^  C, -h  MF. 
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The  left-hand  side  represents  the  end  of  a  search  period  of  duration  X  minutes  and  the  right- 
hand  side  the  beginning  of  a  no-search  period.  A  second  analogous  equation  can  be  set  up  for 
the  end  of  a  period  of  no  search  and  the  beginning  of  a  period  of  search.  These  equations  are 
then  solved  to  find  C|  and  C3  as  functions  of  X.  The  expected  target  level  in  each  of  the 
regions  at  any  instant  in  time  can  now  be  written  as  an  integral  function  of  only  one  parameter, 
X,  the  half-cycle  length,  which  yields 

(6)  ST  =  -  iC^KlFX))  X  (1  -  exp  i-FX))  -F  AflF 

~  {CiUiF  +  DX))  X  (1  -  exp(-(f  -F  l)X)  -F  MKF  -F  I). 

The  average  for  the  entire  region,  S,  is  merely  twice  that  of  any  individual  region. 

With  some  algebraic  manipulation  of  the  derivative,  we  have  proved  [4]  that  S  increases 
monotonically  as  X  increases.  In  other  words,  the  average  expected  target  level  in  each  region 
decreases  as  the  frequency  of  transfers  between  the  two  regions  increases.  This  result  is 
directly  comparable  to  that  of  Gilbert  {6|.  This  result  also  implies  that  if  search  were  scheduled 
in  long  blocks  of  time  (i.e.,  increase  AO,  and  no  penalties  were  incurred  for  switching  from 
search  to  no  search  the  model  predicts  a  decrease  in  the  probability  of  interception. 

Magnitude  of  Impact  of  Short  Cycles 

To  determine  the  magnitude  of  improvement  produced  by  the  shorter  cycles,  we  deter¬ 
mined  the  average  number  of  targets  present  as  the  cycle  length,  X,  approaches  infinity. 

(7)  5T  =  \I2UIF)  -F  mUKF  +  1))^  A  (IF  +  /)/2f  (f  +  /) 

For  short  cycles  as  Af  approaches  zero.  Equation  (6)  becomes  2Al(2F  +  I)  and  thus  the  reduc¬ 
tion  of  the  average  target  level  for  short  cycles  as  compared  to  long  cycles  is 

(8)  \I(2FII +  \)‘. 

This  expression,  not  surprisingly,  does  not  depend  on  the  arrival  rate.  A,  and  depends  only  on 
the  ratio  of  the  departure  rate,  F,  to  the  interception  rate,  /.  Thus,  for  example,  if  the  depar¬ 
ture  and  detection  rates  were  equal,  infinitesimally  short  cycles  would  reduce  the  target  level  by 
1/9  as  compared  to  long  cycles.  Table  I  summarizes  the  reduction  for  a  range  of  ratios.  As  the 
departure  rate  grows  (shorter  mean  duration)  relative  to  the  interception  rale,  the  impact  of 
shorter  cycles  decreases  rapidly. 

If,  as  an  alternative,  we  measure  the  relative  impact  of  long  and  short  cycles  against  a  no 
search  policy  then  the  reduction  due  to  infinitely  long  searches  is  //2(F  -F  I)  and  for  short 
cycles  it  is  l/(2F  -F  /).  If,  for  example,  /  =  Flhen  short  cycles  reduce  the  target  level  by  1/3 
more  than  long  cycles.  The  probability  of  interception  under  each  policy  is  directly  related  to 
this  proportional  reduction  in  target  level.  For  infinitely  long  cycles  the  fraction  of  targets  inter¬ 
cepted  while  searching  is  just  l/(F  -F  /)  but  we  search  only  half  of  the  time,  (l/2(F  -F  I). 

For  infinitesimally  short  cycles  during  any  target's  lifetime,  the  searcher  and  target  will  be 
in  the  same  region  half  of  the  time.  Consequently,  the  probability  of  interception  is 

(9)  f  F  exp  (-Ft)  X  (1  -  exp(-lt/2))  dt. 

This  yields  l//(2F  -F  /)],  which  represents  the  reduction  in  target  level  for  infinitesimally  short 
cycles. 
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TABLE  1  —  A  Comparison  of  Short  and  Long  Cycles  With 
Regard  to  Interception  Probabilities  and  the 
A  verage  Expected  Number  of  Targets  Present 


R  =  F/l 

Reduction  in 
Target  Level 

Hue  to  Short  Cycles 

Probability  of  Interception 

1 

I  -  SJS, 

All 

Short  Cycles 
.333 

Long  Cycles 
.250 

2 

.040 

.200 

.167 

i 

.020 

.143 

.125 

4 

0123 

111 

.100 

5 

0083 

.091 

,083 

10 

.0023 

.0476 

.0455 

20 

.00059 

.0244 

.0238 

30 

.00027 

.0164 

.0161 

40 

.00015 

.0123 

.0122 

50 

.000098 

.0099 

.0098 

4.  TWO  REGIONS  -  EQUAL  -  TARGET  RATES  -  TIME  LOST  IN  TRANSFER 

In  this  section  we  modify  the  previous  example  by  introducing  a  parameter  L,  which 
represents  the  time  lost  from  search  while  traveling  between  the  two  regions.  We  will  use  the 
differential  equation  model  to  address  two  questions. 

1 .  What  is  the  optimal  value  of  X,  the  time  spent  in  R 1  before  switching  to  R2?  The 
optimal  value  is  obviously  no  longer  infinitesimal. 

2.  Is  there  a  simple  analytic  expression  which  specifies  for  which  values  of  L  it  does  not 
even  pay  to  switch  regions? 

In  tackling  the  problem  with  the  differential  equation  model,  we  will  again  assume  a  cycli¬ 
cal  strategy  (both  Kisi  [81  and  Gilbert  (6l  have  cyclic  strategies)  and  that  the  search  effort  will 
be  divided  equally  between  the  two  regions. 


The  key  to  our  analysis  is  that  optimal  solutions  which  limit  the  search  to  only  one  region 
will  appear  in  the  analysis  as  solutions  in  which  the  optimal  value  of  X  is  infinite. 


In  this  second  problem,  which  ascribes  a  penalty  for  switching  regions,  the  basic  equations 
which  describe  the  periods  of  search  and  no  search  do  not  vary  from  those  of  the  previous 
example  (Equations  2  and  4).  The  introduction  of  the  switching  time,  L,  does  affect  the  con¬ 
tinuity  boundary  conditions  and  changes  Cj  and  C2  since  a  no  search  period  in  each  region  has 
a  duration  of  'X  +  2L'  minutes.  The  average  51  is  calculated  as  in  Equation  (6)  except  that 
the  no  search  time  period  extends  over  a  time  period  of  X  +  2L  minutes  and  that  the  cycle 
length  is  now  2X  -I-  2L.  The  resultant  expression  is 


(10) 


S\~  A/F-  [AIX/2{X  +  L)F(F  +  /)!  -t-  lA/2(X  -I-  D] 
X  ll/F(F+  nv  X 


I  .  2  -  exp(-(/  +  F)X)  -  exp  (-(AT  +  2L)F) 
ex[>{-{2F  +  l)X -2FL)  -  \ 
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Once  values  have  been  assigned  to  F,  I  and  L,  this  equation  is  a  function  of  only  one  vari¬ 
able,  X,  whose  optimum  is  easily  determined.  In  Figure  1  the  optimum  value  of  X  is  shown  for 
a  range  of  L,  F  and  R,  where  R  is  the  ratio  of  F,  the  departure  rate,  to  /,  the  interception  rate. 
There  are  graphs  for  two  values  of  f,  one  for  /'equal  to  20— targets  last  an  average  of  three 
minutes— and  one  for  Fequal  to  10,  with  R  as  high  as  20,  and  as  low  as  .01,  which  would  mean 
that  targets  are  intercepted  at  a  rate  one  hundred  times  as  fast  as  they  depart  on  their  own.  The 
lower  values  for  R  were  included  less  for  realism  than  to  display  how  the  curves  behave  as  R 
approaches  the  limit  of  zero. 


R=  20  10  5  2  1  5  1  01 


The  most  striking  characteristic  of  these  curves  is  their  asymptotic  nature  and  the  location 
of  the  asymptote. 

The  asymptote  is  always  less  than  the  mean  duration  of  the  target,  \/F,  and  approaches 
the  mean  duration  as  R  approaches  0.  In  addition,  for  a  given  value  of  L  as  the  ratio,  R, 
decreases,  the  optimum  X  also  decreases;  as  F  decreases  (from  20  to  10)  the  value  of  the 
asymptote  for  a  given  R  increases. 

All  of  the  above  points  will  be  addressed  more  formally  in  the  succeeding  section  in  which  an 
analytic  expression  for  the  asymptote  as  a  function  of  f  and  R  is  derived. 

When  to  Search  Only  One  Region 

The  development  of  an  analytic  expression  for  the  asymptote  as  a  function  of  Fand  R 
builds  on  the  observation  that  51,  the  average  target  level  will  either  be  (1)  a  monotonically 


VOL  28.  NO  1,  SEPTEMBER  1981 


NAVAL  RESEARCH  LOGISTICS  QUARTERLY 


414 


K.  R.  CHELST 


decreasing  function  of  X,  or  (2)  a  unimodai  function.  We  have  no  formal  proof  of  this  but 
extensive  computer  analysis  of  SI  for  a  wide  range  of  values  of  F,  R  and  X supports  this  claim. 
In  both  cases  the  limiting  value  as  X  tends  to  infinity  is  the  same.  Because  of  the  complexity  of 
the  derivative,  we  found  that  the  simplest  way  to  determine  monotonicity  was  to  explore  the 
limiting  behavior  of  the  derivative.  If  the  derivative  approached  zero  from  the  negative  direc¬ 
tion  as  X  increased  to  infinity  then  the  function  was  monotonic  and  the  optimal  strategy  was  to 
search  only  one  region.  It  can  be  shown  that  after  excluding  all  components  of  the  derivative 
which  approach  zero  faster  than  \I{X  +  L)^  and  consider  only  the  direction  of  the  derivative, 
we  find  the  expression  of  interest  to  be 


(11) 


lim  -  L  - 

X— oo 


F(F  +  I)  ^ 


—  exp  (-(/  +  F)x)  —  exp  {-F{1L  + 


with  D  =  exp(-2f(;ir  -t-  L)  -  /Jf)  -  1. 


Its  limit  is 

(12)  -  L-  II FU  +  F)  +  2I/FU  +  F)  =  -  L  +  l\/F]  x  ll/((f//)  -t-  1)]. 
This  will  be  negative  and  thus  the  optimal  value  of  X  infinite  whenever 

(13)  L  >  ll/F]  X  ll/((f//)  1)1. 


This  confirms  our  earlier  graphical  analysis.  The  second  component  [IIHF/D  +  1)1  is 
always  less  than  1  and  approaches  1  when  R,  the  ratio  of  F  to  /,  approaches  0.  Thus,  the 
asymptote  for  each  curve  in  Figure  1  is  less  than  the  mean  target  duration,  1/f,  and  approaches 
the  mean  target  duration  as  R  approaches  0.  Of  perhaps  greater  significance  is  that  even  if  L  is 
orders  of  magnitude  shorter  than  the  mean  duration,  the  optimal  strategy  might  be  to  search 
only  one  region.  For  example,  assume  targets  depart  at  nine  times  the  rate  they  are  detected. 
If  the  travel  time  is  more  than  one-tenth  the  average  duration  of  a  target  (e.g.,  12  seconds  for 
two  minute  targets,  18  seconds  for  three  minute  ones),  the  optimal  strategy  is  to  search  only 
one  region. 

5.  DIFFERING  TARGET  ARRIVAL  RATES 

The  final  example  involves  two  regions,  RH  and  RL.  In  RL  targets  arrive  at  a  rate  A  and 
in  RH,  the  high  target  region,  targets  arrive  at  a  rate  M  x  A,  with  M  greater  than  I.  In  both 
regions  all  targets  have  the  same  departure  and  interception  rates,  F  and  /,  respectively.  Once 
again,  our  analysis  will  revolve  about  cyclic  policies,  this  time  of  the  form,  X  minutes  in  RL 
followed  by  A  X  A'  minutes  in  RH. 

Although  one  problem  of  interest  is  the  finding  of  the  (A,  X)  pair  which  minimizes  the 
average  expected  number  of  targets  in  progress,  the  discussion  will  not  be  limited  to  that,  since 
in  all  instances  the  optimum  is  approached  as  X  tends  towards  zero,  an  unimplementable 
optimum.  Therefore,  the  development  will  also  address  the  issue  of  the  optimum  value  of  A 
for  a  given  value  of  X.  Setting  A  to  be  a  specified  value  is  interpretable  as  establishing  a  feasi¬ 
bility  constraint  on  the  search  process  in  RL.  We  have  not  constrained  A  to  be  greater  than  1 . 
If  A  is  less  than  one,  less  search  time  is  spent  in  the  high  target  region. 
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The  discussion  that  will  follow,  then,  can  be  categorized  briefly  as; 

1 .  A  single  expression  is  developed  to  be  used  in  obtaining  the  optimal  value  of  K  for  a 
given  X. 

2.  An  analytic  expression  is  found  for  the  optimal  K  value  as  X  approaches  zero. 

3.  Evolving  directly  from  2,  is  an  expression  for  identifying  values  of  M  for  which  the 
optimal  K  value  is  infinite  for  all  values  of  X.  (If  the  optimal  K  value  is  infinite  then 
the  optimal  strategy  limits  search  to  the  high  target  region.) 

4.  Lastly,  an  expression  is  found  which  specifies  the  values  of  as  a  function  of  M,  f, 
and  /,  for  which  the  optimal  K  is  again  infinite. 

In  this  example,  the  regions  are  not  symmetric  and  separate  differential  equations  are 
needed  for  each  region.  The  equations  for  the  expected  number  of  targets  in  the  low  target 
region  are  the  same  as  before.  Equations  (2)  and  (4).  The  differential  equations  for  the  high 
target  region  and  their  solutions  are  also  basically  the  same  as  before  except  for  replacing  >4,  the 
arrival  rate,  by  M  x  /l.  We  then  set  up  the  standard  boundary  conditions  which  constrain  the 
target  level  to  be  continuous  and  solve  for  their  constants,  C|,  C2,  Cj,  and  C4.  In  this  case  a 
cycle  consists  of  KX  minutes  of  search  in  RH  followed  by  X  minutes  of  search  in  RL. 

The  average  expected  number  of  targets  present  in  the  combined  two-region  area  is  then 
determined  with  the  following  equation 

^  “  Ik  IiTx  X*"  ^ 

+  C4  exp(-(f  +  /)/)  +  MAlil  +  F)]dt 

+  A/U  +  F)  +  C,  expi-Fi)  +  MA/F]dt. 


The  first  integral  represents  the  average  number  of  targets  in  both  regions  during  the 
search  (KX  minutes)  of  RH  and  the  s«:ond,  the  average  during  the  search  of  RL.  After 
integrating  out,  substituting  where  necessary,  and  combining  terms  wherever  possible,  the 
result  is 


MA  .  A  + 
F  +  I  F 


AliM  -  1) 

{K  +  l)f(f  +  /) 


(K  +  DJr 


AP 

fHf  +  Ip 


M{\  -  exp(-FAr))  (1  -  exp(-(F  +  DKX)) 
1  -  exp(-(F(K  +  1)  +  /X)X) 

(1  -  exp(-FAX))  (1  -  exp(-(F  +  I)X)) 

\  -  exp(-(F(K  +  ])  +  I)X) 
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Our  unaiysis  ol  ibis  expression  is  delayed  until  later;  for  now  we  present  just  a  graphical 
anaivsis.  In  f  igures  2  and  3  we  have  graphed  the  optimal  value  of  /C  as  a  function  of  X,  the 
duraiiun  ol  a  visit  to  RL  In  each  case  targets  are  of  a  5  minute  duration,  f  ==  12.  Figure  3 
graphs  the  rclaii nnslnp  lor  a  range  of  M  (the  ratio  of  high  to  low  region  target  rates)  with  the 
rati'.  /  !')  /  set  a!  1  A  common  characteristic  of  the  curves,  except  for  the  M  equal  to  one 
cui  .i ,  \s  that  as  A  increases,  the  optimal  K  value  grows  asymptotically  to  infinity  (i.e.,  search 
')iil>  one  isgionl  Not  surprisingly,  as  M  increases  the  asymptote  decreases.  As  the  difference 
in  target  .irrival  rates  increases,  the  optimal  strategy  is  more  and  more  likely  to  focus  only  on 
the  high  target  region. 


M  =  2 


F  =  12 
R  =  F  I  =  1 


1  0  K' 


M  =  1 


I  lot  Ki  2  t  he  oplimal  vatue  of  K  as  a  function  for  a  range  of  W  values 


R  =  4  3 


M  =  1  2 


1  I  a  4  hrs  X 

I'Kit  RK  .r  t  he  optimal  value  of  A'  as  a  function  of  X  for  a  range  of  values  of  R 


Figure  3  graphs  K  as  a  function  of  X  for  a  range  of  R  with  M  set  at  1 .2.  As  R  increases, 
the  cuiofl  value  of  X,  above  which  only  RH  \%  searched,  decreases.  For  /?  -=  1  the  cutoff  value 
of  X  is  approximately  .43  hours  and  as  R  increases  to  4  this  becomes  .18  hours. 
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One  characteristic  of  the  curves  that  seems  counterintuitive  is  that  as  X  increases  initially, 
the  optimal  value  of  K  decreases.  A  corollary  of  this  is  that  the  optimal  stiategy  within  the 
constraints  we  have  imposed  may  allocate  less  than  half  the  search  effort  to  the  high  target 
region.  For  identical  regions  the  optimal  strategy  divides  the  search  equally  only  in  the  limit  as 
X  goes  to  zero. 

There  seem  to  be  two  conflicting  forces  at  work.  The  more  intuitive  one  is  that  as  \ 
increases  the  searcher  is  forced  to  incur  increased  diminishing  return  in  any  visit  to  the  low  tar¬ 
get  region  and  eventually  the  cost  for  visiting  the  region  RL  becomes  so  prohibitive  that  it  no 
longer  pays  to  search  the  region.  This  force  dominates  in  the  long  run.  However,  there  is  an 
advantage,  in  general,  to  having  short  cycles  and  as  X  increases  from  0  there  is  only  one  way  of 
restraining  the  increase  in  cycle  length  and  that  is  by  decreasing  K.  This  propensity  for  shorter 
cycles  seems  to  dominate  the  behavior  of  the  optimal  value  of  K  for  small  values  of  A'  turning 
K  optimal  initially  into  a  decreasing  function  of  A". 

No  Constraint  on  the  Duration  of  a  Search 


In  this  section  we  focus  on  the  limiting  behavior  of  S’ as  A"  approaches  zero  ti  e.,  no  con¬ 
straint  on  the  minimum  duration  of  a  visit  to  a  region).  Although  as  X  approaches  zero  the 
optimal  solutions  are  no  longer  implementable^  the  analysis  of  the  limit  properties  will  prove 
signihcani,  because  the  absolute  minimum  of  S  is  always  approached  as  X  goes  to  zero.  The 
limit  of  S  as  AT  approaches  zero  is 

c  ^  MA  .  A  ■  AIM  _  _  A I _ _ 

1-0  I  +  F  F  (F  +  1)  iF(K  -^  \)  +  IK)  FiFiK 
The  optimal  value  of  K  is 


(17) 


K  (optimal)  = 


+2R{R  +\){M  -  \)  ±  2(2R  +  DM'- 
2(R  -F  1)2  -  2MR" 


where  F/ 1.  It  is  again  clear  that  the  optimal  value  for  K  does  not  depend  upon  the 
independent  values  of  Fand  /  but  only  on  their  ratio,  thereby  reducing,  once  again,  the  number 
of  critical  parameters.  Table  2  displays  the  optimal  K  for  a  range  of  M  and  R.  For  M  equal  to 
one,  which  means  that  the  two  regions  generate  targets  at  the  same  rate,  the  optimal  value  of  K 
does  not  depend  upon  R  but  is  always  one  (i.e.,  equal  search  in  both  regions),  as  was  assumed 
earlier  in  this  paper. 


One  aspect  of  the  Equation  (17)  that  needs  clarification  is  that  since  there  are  two  solu¬ 
tions  is  it  possible  for  both  solutions  to  be  positive?  As  it  turns  out,  this  can  never  happen. 
For  the  range  of  M  which  results  in  both  values  of  the  numerator  being  positive,  the 
corresponding  denominator  will  be  negative.  If  the  numerator  is  negative  the  denominator  is 
positive.  This  last  result  provides  the  key  to  finding  under  what  conditions  the  optimal  solution 
is  to  search  only  one  region. 


Search  Only  One  Region 


Whenever  both  solutions  are  negative,  for  M  greater  than  one,  the  optimal  solution  will 
be  at  the  upper  bound  of  the  feasible  region,  namely  K  optimal  will  be  infinite  (i.e.,  search  only 
one  region).  Consequently,  the  optimal  search  focuses  on  only  one  region  whenever 
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TABLE  2  —  The  Optima!  yalue  of  K  as  a  Function  of  M  and  R  —  No 
Constraint  on  Minimum  Duration  of  a  Fisit  to  RL  (X  —  0) 


M 

.1 

.5 

1 

2 

5 

10 

20 

30 

40 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1.1 

1.06 

1.10 

1.15 

1.27 

1.71 

3.00 

85.0 

OO 

OO 

1,2 

1.12 

1.20 

1.32 

1.59 

3.01 

45.0 

OO 

OO 

OO 

1.3 

1.17 

1.30 

1.49 

1.97 

6.16 

OO 

OO 

OO 

OO 

1.4 

1.22 

1.40 

1.67 

2.45 

25.0 

OO 

OO 

OO 

OO 

1.5 

1.28 

1.51 

1.87 

3.04 

OO 

OO 

OO 

OO 

OO 

2 

1.52 

2.05 

3.12 

13.1 

OO 

OO 

OO 

OO 

OO 

2.5 

1.74 

2.64 

5.16 

OO 

OO 

OO 

OO 

OO 

OO 

3 

1.95 

3.31 

9.20 

OO 

OO 

OO 

' 

OO 

OO 

OO 

3.5 

2.15 

4.09 

21.2 

OO 

OO 

OO 

OO 

OO 

OO 

4 

2.33 

5.00 

OO 

OO 

OO 

OO 

OO 

OO 

OO 

(18) 


M  > 


{R  +  1)^ 
R^ 


To  be  rigorous,  we  showed  only  that  in  the  limit  as  X  approaches  zero  that  the  inequality 
specifies  whether  or  not  the  low  target  region  should  be  searched.  However,  if  it  does  not  pay 
to  ever  search  the  low  target  region  for  an  infinitesimally  short  duration  there  should  be  no 
incentive  to  search  it  if  the  minimum  duration  of  a  visit  is  instead  some  number  larger  than 
zero. 


In  order  to  emphasize  the  impact  of  this  result  we  display  in  Figure  4  the  convex  region  in 
R,  M  space  for  which  the  optimal  strategy  limits  the  search  to  RH.  For  example,  if  R,  the  ratio 
of  departure  to  interception,  were  10  then  if  /{A/ generated  21  percent  more  targets  than  RL, 
only  RH  would  be  searched.  If  R  is  20,  30,  or  40,  then  if  RH  generated  respectively  10%,  7%, 
or  5%  more  targets  than  RL,  only  the  high  target  region  should  be  searched.  The  result  is  a 
second  strong  limitation  on  the  likelihood  of  searching  more  than  one  region. 

In  the  preceding  analysis,  we  showed  that  for  given  values  of  f  and  /,  if  the  ratio  of  target 
arrival  rates  is  above  some  cutoff  value,  M,  the  optimal  strategy  is  to  search  only  RH  even  for 
infinitesimally  small  values  of  X.  However,  even  for  values  of  M  below  the  cutoff  if  X,  the 
time  to  search  RL,  is  'too  large,'  the  optimal  strategy  will  again  restrict  the  search  to  RH. 
Therefore,  the  next  question  to  resolve  is,  "When  is  X  too  large?" 


In  Equation  (15)  only  two  components  are  functions  of  K,  with  only  the  negative  com¬ 
ponent  a  function  also  of  X.  The  two  components  are 


(19) 


_J A/(M  -  1) 

K  +  l  F(F  +  /) 
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Figure  4.  A  characterization  of  when  to  search  only  high  target 
region  as  a  function  of  two  parameters 


(20) 


-1 


K  +  1 


xfHf  +  /)2 


X  Mil  -  exp(-FAf))  X  -  +  OKX)) 


(1  -  exp(-(r(Ar  +  1)  +  /)x)) 


+  (1  -  exp(-(f  +  /)X))  X 


(1  -  exp(-fO)) 

(1  -  exp(-(F(/(  +  1)  +  /)X)) 


If  for  some  value  of  X  the  magnitude  of  the  first  component  were  greater  than  that  of  the 
second  for  all  values  of  K,  then  their  sum  would  always  be  positive.  The  optimal  strategy  then 
would  be  to  allow  K  to  go  to  infinity,  since  that  would  drive  each  component  to  zero. 

In  comparing  their  relative  magnitudes,  we  ignore  the  common  factor  l/(K  +  1);  this 
turns  the  second  expression  into  a  monotonically  increasing  function  of  fC.  Its  limit,  as  /C 
grows  without  bound,  is 

(21)  X  [Afd  -  exp(-f20)  +  1  -  exp(-(F  +  l)X)]. 

XF^(F  +  /)■' 

The  problem  therefore  reduces  to  finding  values  of  X  such  that 

(22)  X  >  -  \/fr(F  +  'l)  “  exp(-FA'))  +  1  -  exp(-(f  4  i)X)]. 


In  Figure  5  we  have  graphed  the  lower  bound  on  X  for  a  range  of  M  values  with  F  and  / 
set  equal  to  1.  A  conservative  (sufficient  but  not  necessary)  bound  on  X  can  be  easily  found 
since  (1  -  expi-FA"))  and  ll  -  exp(-(F  4  /)A')1  are  both  bounded  by  i.  This  bound  is 


(23) 


X  > 


A/4-  1  ^ 
M-  1 


/ 

F(F  -F  /) 


which  approaches  the  actual  bound  as  M  approaches  I.  See  Figure  S  for  a  comparison  with  the 
actual  lower  limits. 
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I'Kii  Ri  5.  Exact  and  conservative  estimates  of  the  cutoff 
vafues  for  .f  for  a  range  of  M.  If  .f  is  above  the  cutoff, 
search  is  limited  to  the  high  target  region 


6.  EXTENSIONS  AND  SUMMARY 


Multiple  Target  Types 

In  order  to  indue  the  multiple  target  types  in  the  model,  it  is  necessary  to  add  for  each 
target  type  different  values  of  /  and  F  and  a  pair  of  differential  equations  for  each  region.  To 
calculate  the  slate  of  the  system,  S,  under  a  particular  search  strategy,  each  of  the  targets  is  ini¬ 
tially  treated  separately.  The  total  average  number  of  targets  in  progress  is  then  just  the  sum  of 
all  the  individual  averages.  The  resultant  expression  is  still  a  function  of  only  X  ox  K  and  X, 
and  can  be  analyzed  in  the  same  manner  as  before.  To  illustrate  this  without  going  into  much 
detail,  we  generalize  our  earlier  results  which  characterize  when  to  search  only  one  region. 


The  first  expression  we  derived  involves  two  identical  regions  with  time  lost,  L,  in  travel¬ 
ing  between  the  regions.  We  associate  with  each  target  type,  y,  three  parameters,  and  A,\ 
the  target  specific  interception  rate,  departure  rate  and  arrival  rate,  respectively.  The  resultant 
expression  for  L  is  now 


(24)  L  ^  - 

/ 

which  unlike  the  earlier  expression  (13)  does  depend  upon  the  arrival  rates. 


f,(F,  +  /,) 


A 


F\F,  +  /,) 


The  last  two  expressions,  one  for  M,  the  other  for  T,  relate  to  the  problem  of  allocating 
search  between  a  high  and  low  target  region.  When  we  generalized  this  problem  and  allowed 
lor  multiple  target  types,  we  also  indexed  the  ratio  of  the  two  region  target  arrival  rates,  Wy, 
and  made  it  target  type  specific.  The  expression  which  corresponds  to  Equation  (18)  and 
specifies  when  to  search  only  the  high  target  region  even  for  small  values  of  X  is 


(2S) 


I 


A,  I, 


(F,  +  ly 


±!l 

F} 


If  the  above  inequality  is  not  satisfied,  large  values  of  X  will  still  restrict  search  to  the  high 
target  region.  The  conservative  form  of  this  equation  with  multiple  target  types  is 
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More  Than  Two  Regions 

Introducing  more  than  two  regions  into  the  problem,  significantly  increases  the  complexity 
of  analysis.  Although  any  of  the  earlier  three  inequalities  can  be  applied  pairwise  to  all  the 
regions  to  exclude  as  many  as  possible,  the  complications  enter  when  more  than  two  regions  arc- 
still  left.  The  heart  of  the  problem  lies  in  the  difficulty  of  defining  cycle  types  to  be  anaiy/cd 
The  work  by  Barnett  (1)  which  tackles  problems  involving  more  than  two  regions  can  vield  n- 
insight  here.  In  his  work  any  search  of  a  region  discovers  all  targets  present  with  probahilit,  i . 
The  consequence  of  this  is  that  after  one  unit  of  search  in  a  region,  diminishing  return  Horn  a 
continuous  search  stops,  which  is  not  the  case  here. 

Summary 

In  this  paper  a  differential  equation  model  of  a  search  process  was  developed  which  has 
the  potential  for  capturing  the  dynamics  of  a  sequential  cyclic  search  strategy  In  displaying  the 
application  of  the  model  to  a  number  of  examples,  we  have  derived  a  number  of  independent 
quantifiable  constraints  which  limit  the  number  of  regions  lo  be  searched  and  which  are  sum¬ 
marized  in  Table  3.  The  main  ihrust  of  all  our  results  strongly  emphasize  that  witii  randomly 
arriving  and  departing  targets  and  travel  times,  search  should  be  limited  lo  f.nlv  '1,.;  i,i,.b,.,,  lar¬ 
ge!  region. 


TABLE  3  —  Summary  oj  Conditions  Unaer  Which 
Only  One  Region  is  Searched 
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ABSTRACT 

We  investigate  a  class  of  optimal  stopping  problems  for  dynamical  systems  described 
by  one-dimensional  differential  equations  with  an  additive  Poisson  disturbance.  The  rate 
of  the  disturbance  may  depend  upon  the  current  stale  of  the  system.  A  dynamic  pro¬ 
gramming  equation  for  the  optimal  stopping  cost  is  derived  along  with  conditions  which 
must  be  met  at  the  boundary  of  the  optimal  stopping  set.  These  boundary  conditions 
depend  upon  whether  or  not  the  stopping  set  may  be  entered  by  smooth  motion. 


1.  INTRODUCTION 

DifTerentiai  equations  driven  by  Poisson  disturbance  terms  appear  in  models  of  the  con¬ 
tents  of  dams  and  inventories  (Prabhu  (71),  viral  growth  (Bartoszyhski  111),  population  dynam¬ 
ics  in  random  environments  (Hanson  and  Tuckwel)  |6]),  collective  risk  (BQhImann  [2]), 
machine  failure  (Wickwire  (111),  and  mass  service  systems  (Prabhu  181),  among  others.  In 
some  of  these  models  the  magnitudes  of  the  disturbances  are  random  variables  (the  disturbance 
term  is  a  compound  Poisson  process),  and  in  others  the  disturbances  have  constant  magnitude; 
the  latter  case  is  often  a  useful  approximation  to  the  former,  and  its  mathematical  treatment  via 
the  theory  of  retarded  differential  equations  differs  significantly  from  the  more  general  approach 
taken  for  distributed  magnitudes.  Many  of  these  models  describe  changes  in  resources  or  popu¬ 
lations  which  lend  themselves  in  a  natural  way  to  the  application  of  control  theory:  dam  con¬ 
tents,  insurance  reserves,  and  inventories  must  satisfy  demands  but  not  be  depleted.  Queueing 
systems  should  serve  as  many  customers  as  possible  consistent  with  running  costs;  latent 
machine  failures  ought  to  be  detected  early  without  a  large  number  of  false  alarms,  and  malig¬ 
nant  viral  growth  must  be  checked  at  acceptable  co.«ts  for  treatment.  In  this  paper  we  shall 
assume  that  Poisson  disturbances  of  a  one-dimension  'vnamical  system  are  of  constant  mag¬ 
nitude  and  shall  also  suppose  that  the  allowal'  >ntr. .  actions,  at  any  time,  are  of  a  simple 
type;  one  may  either  stop  the  system  and  gaiu  '  iiich  depends  upon  the  current  value 

of  the  system,  or  let  it  continue  to  evolve  (in  tne  hO|^  that  the  reward,  at  a  subsequent  time, 
will  be  greater  than  its  current  value) .  Such  problems  are  called  optimal  stopping  problems  and 
their  general  theory  has  recently  attracted  much  attention  (see  Chow,  Robbins  and  Siegmund 
[31  for  the  case  of  stopped  random  sequences  and  Siryaev  [91  for  the  case  of  stopped  Markov 
sequences  and  random  processes).  A  treatment  of  optimal  stopping  problems  which  can  be 
applied  to  mixed  Markov  processes  (that  is,  those  whose  sample  paths  have  jumps  superim¬ 
posed  on  continuous  movement)  has  been  given  by  Grigelionis  [Si,  but  his  most  important 
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result,  the  derivation  of  conditions  satisfied  by  the  payoff  function  at  an  optimal  sloping  boun¬ 
dary,  does  not,  apparently,  include  the  case  of  a  process  which  can  cross  such  a  boundary  only 
by  means  of  a  jump.  We  shall  derive  here  a  set  of  conditions  satisfied  by  the  payoff  and 
optimal  stopping  boundary  which  applies  to  both  smooth  and  discontinuous  entry  into  the  set  of 
states  at  which  stopping  is  optimal.  We  shall  limit  ourselves  to  autonomous  dynamics  defined 
on  a  finite  interval,  because  this  case  exhibits  the  essential  features  of  a  wider  class  of  prob¬ 
lems.  We  remark  that  optimality  conditions  for  problems  with  control  actions  more  general 
than  those  considered  here— actions  which  alter  paths  without  stopping  them— may  often  be 
derived  in  an  analogous  way. 

2.  FORMULATION  OF  THE  STOPPING  PROBLEM 


Let  lx,!,  /  ^  0,  be  a  completely  observable  Markov  process  on  the  probability  space 
[E.  P],  where  £  c  /?'.  Since  we  shall  consider  £  in  what  follows  to  be  a  finite  interval,  no 

generality  is  lost  by  taking  £  =  10,  ^  <  o«>,  and  we  shall  do  so.  Suppose  that  x,  satisfies  the 

stochastic  differential  equation  (Gihman  and  Skorohod  I4l) 

(1)  dx,=  a(x,)dt  +  J*  |/3(x,.  u)N(dt.  du).  xo=  x. 

where  ai  )  is  continuous  on  £  and  nonzero  on  £  —  ({O!  U  {^1).  (The  arguments  below  can  be 
modified  without  difficulty  to  account  for  functions  a(  )  which  vanish  inside  £.)  The  function 
^  (x.  m),  the  magnitude  of  the  disturbance,  is  defined  as 


(2) 


«  if  u  ^  \  '(.v) 
0  otherwise. 


where  let  is  a  bounded  constant  and  X(x)  >  Xo  >  0  ffic  rate  at  which  disturbances  occur. 
/V(f,  ■)  is  a  random  Poisson  measure  in  R'  such  that  for  any  Borel  set  ^4  C  /?',  £  W(/,  /f)  = 
f;r(/4).  where  iris  a  finite  measure  which  assigns  mass  X(x)  to  the  interval  IX'Ux'),  <»).  Solu¬ 
tions  of  Equation  (1)  satisfy  the  ordinary  differential  equation  dx! dt  -  aix)  between  distur¬ 
bances  of  magnitude  « ,  and  the  latter  occur  according  to  a  Poisson  process  with  rate  X  (x,).  The 
infinitesimal  generator  A  of  {x,|,  which  acts  on  bounded,  continuous  functions  f.E—  R\  is 
defined  by  Af(x)  =  a(x)/’(x)  +  X  (x)  l/(x  -I-  e)  -  /(x)].  The  behavior  of  a(x)  and  the  sign 
of  «  will  determine  the  boundary  conditions  which  may  be  imposed  at  x  =  0  or  x  =  ?  upon 
solutions  of  Af(x)  =  bf(x),  an  equation  which  will  occur  below. 


Let  g(  )  ^  0  be  a  bounded,  continuous  function  on  /?',  and  J(\\\e  set  of  stopping  times 
relative  to  the  family  of  o-  -  algebras  ^  C  .^generated  by  {x,l.  The  optimal  stopping  problem 
we  shall  consider  is  to  find  a  stopping  time  t  M  (if  one  exists)  such  that  for  any  x  €  £ 

(3)  gixz)  =  sup  g(x^)  =  s(x),  h  >  0, 

T  ^ 

where  £,  denotes  the  expectation  conditional  on  xq  =■  x.  In  the  next  section  we  shall  derive  a 
dynamic  programming  equation  satisfied  by  the  payoff  j(x). 


3.  A  FUNCTIONAL  EQUATION  SATISFIED  BY  six) 


Suppose  that  xq  *  x  and  we  wish  to  follow  an  optimal  stopping  policy  for  ail  future  time. 
If  we  stop  the  process,  we  gain  gix)  as  payoff;  if  we  decide  to  wait  a  time  A  >  0  for  a  higher 
payoff,  our  expected  gain  is  £,  j(x  4-  Ax),  where  Ax  is  the  distance  x,  has  moved  during 
A.  Since  we  may  only  wait  or  stop  the  process,  we  have 
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(4)  s(,v)  =  max  (^(x).  £,  s(.v  + 

and  if  six)  6  the  domain  of  /I,  £,e-'’^s(x  +  Ax)  =  sU)  +  A/<5(x)  -  bSs(x)  +  o(A), 
whence 

six)  =  max  lf(ix),  six)  +  A/4s(x)  -  dAsix)  +  o(A)|, 
or 

(5)  0  =  max  k(.v)  -  5(x).  A(/ls(x)  -  bsix))  +  o(A)). 

Since  A  >  0  is  arbitrary,  we  have  the  dynamic  programming  equation 

(6)  max  lg(x)  -  s(x).  4s(x)  -  bsix)]  =  0,  s  6  £)^. 

if  y  is  the  set  of  points  in  E  at  which  it  is  optimal  to  stop,  and  ^  -  £  -  yis  the  optimal  con¬ 
tinuation  set,  we  have  from  Equation  (6)  that 

(7)  s(x)  =  ^(x)  and  Asix)  <  bsix)  <— >  x  6  y 

and  4s(x)  =■  bsix)  and  s(x)  >  g(x)  <=»=>  x  €  <i^  . 

By  Theorem  6,  p.  108,  of  Siryaev  [9],  f,  the  optimal  stopping  time,  is  equal  to  inf 
{/  >  0:  X,  €y),  provided  that  PH  <  <»)  =  1.  The  equations  at  (7)  are  not  enough  to  deter¬ 
mine  an  optimal  policy  (and  therefore  six))  because  they  do  not  specify  the  boundary  of  y 
The  next  section  is  devoted  to  a  derivation  of  condtions  for  "matching"  six)  to  gix)  at  the 
boundary  of  y  These  conditions  will  serve  to  determine  an  optimal  policy  uniquely  in  most 
cases  of  practical  interest. 

4.  OPTIMAL  MATCHING  CONDITIONS  AT  dy 

The  nature  of  these  conditions  depends  upon  whether  x,  crosses  dy  by  smooth  or  discon¬ 
tinuous  movement.  The  following  theorem  treats  the  case  of  continuous  movement  and  is  part 
of  the  lore  of  optimal  stopping  for  various  continuous  processes.  It  says,  in  our  case,  that  if  x, 
can  enter  yby  smooth  movement,  the  payoff  six)  will  be  a  smooth  function  on  Sy inside  £. 

THEOREM  1;  Let  xo=<r  €  d.y,  <r  0,  and  suppose  that  for  any  small  A  >  0, 
x,j  €  y  if  there  is  no  jump  during  10,  Al.  Assume  further  that  Dix)  =  5(x)  -  g(x)  is  in 
at  X  =  O’,  s'ia-)  and  gTo")  exist  and  are  continuous  with  the  derivatives  taken  for  x  — 'o-  from 
within  Then 

dsicr)  dgi<r) 

where  dldx  \s  the  left  (right)  derivative  if  o(o-)  >  (<)  0. 

PROOF;  (A  modification  of  an  argument  first  used  by  Grigelionis  [5]).  By  hypothesis, 
E„Dix^)  =  A/4  Dix)  4-  o(A)  for  small  A  >  0.  On  the  other  hand, 

E„  Dix^)  =  Dior  +  <7(cr)A)  |l  “  J*  \(m(s))dsj 

+  Di<T  -t-e)  r  kimis))ds  +  o(A) 

■Jo 

=  Dicr  -t-  e)  X(<r)A  +  o(A), 


VOL  28,  NO.  3.  SEPTEMBER  1981 


NAVAL  RESEARCH  LOGISTICS  QUARTERLY 


426 


K,  11,  WICKWIRK 


where  dm  (s)/ds  *  a(m(.s)),  m  (0)  =  <r .  Hence, 

AD((t  )  =  \  (<r)  D  (a-  +  €  ), 
or 

a  ((t)  D'itr)  +  k{a)D{a  +  <)  =  X(cr)D(cr  +  e). 
so  that  (a  (o- )  ^  0) 

s'(<r)  =  g'(<r). 
which  was  to  be  proved. 

The  following  theorem  treats  the  case  in  which  x,  can  enter  yonly  by  means  of  a  jump. 
Grigelionis’  theorem  does  not  apply  here  because  the  expected  exit  time  from  a  neighborhood 
of  o-  into  5^is  bounded  away  from  zero.  The  proof  of  his  theorem  also  requires  that  .v  be  in  the 
domain  of  A  at  cr  ;  since  solutions  of  /l/(x)  =  b/(x),  x  6  /(x)  =  g(x),  x  €  5^,  generally 

have  a  discontinuity  at  a  in  this  case,  it  is  not  a  priori  clear  that  s  satisfies  this  requirement. 
We  shall  see  that  the  requirement  for  matching  of  derivatives  is  replaced  by  that  of  continuity 
of  s  across  d  f^when  x,  can  enter  ^only  by  a  jump. 

THEOREM  2;  Suppose  that  Xo=  o-  €  a  ^  0,  ^ ,  and  that  for  any  small  A  >  0, 
Xi  €  ^  if  X,  has  no  jump  during  (0.  A].  Then  the  following  conclusions  hold: 

(a)  if  o-  +  €  €  ^  and  a  -  e  ^  if,  then  no  optimal  boundary  condition  may  be  assigned  at 

; 

(b)  if  cr  +  €  €  y.  then 

(9)  i(cr±)  =  g(<r)  if  €  $  0, 

where  s(fr  ±)  =  lim  s(x). 

x—v 

X'^<T 

PROOF:  Under  the  conditions  of  (a),  cr  is  not  accessible  on  an  optimal  path  from  'if. 
Consider  (b).  By  the  definition  of  s,  s(<r±)  >  g(x).  Now  suppose  that  instead  of  stopping  at 
cr  (which  is  optimal)  we  let  x,  evolve  for  a  short  time  A  >  0  during  which  x,  moves  a  distance 
Ax.  Then 

g(cr)  ^  Ea  s(cr  +  Axje'"^ 

“  ls(cr  ±  )  +  a  (<r  )As'(cr  )1  (1  —  X  (cr  )A)  (1  —  6A)  +  s((r  +  €  )X  (cr  )  A+  o(A) 

”  A[o(<r  )s'(cr  )  +  X  (cr  )s(<r  +  « )  —  (X  (cr )  +  b)s(<T  ±))  +  s(fT  ±)  +  p(A), 

where  s'  is  the  appropriate  one-sided  derivative.  Since  A  >  0  is  arbitrary,  we  have 
s(cr  ±)  <  g(cr),  whence  the  theorem  is  proved. 

5.  AN  EXAMPLE 

Suppose  that  a  (x)  —  a,  X  (x)  —  X  >  0,  g(x)  —  x^  and  that  0  and  i  are  reflecting  barriers 
if  they  can  be  reached  by  x,.  This  is  a  simple  model  for  a  dam  or  inventory  whose  input  (out¬ 
put)  is  described  by  the  ordinary  differential  equation  dm{s)/ds  -  a,  and  whose  output  (input) 
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is  described  by  a  Poisson  process  with  rate  X  and  jumps  of  size  €.  The  reflecting  barrier 
assumption  implies  that  the  content  of  the  corresponding  system  stays  at  0  (or  ^ )  if  it  reaches 
or  jumps  over  this  level  until  new  input  (or  output)  causes  the  content  to  assume  a  value  in 
(0,  $).  Then  by  Section  3 

s(x)  =  sup 


satisfies 


(10) 


4s (x)  =  bs(x)  in 
six)  =  in y 


where  4s(x)  =  as'(x)  +  X[s(x  +  «)  -  s(x)l.  Using  the  fact  that  h(x)  =  E^e  '’^^(x^)  is 
convex  if  g  is  convex,  one  can  show,  via  Equation  (4),  that  for  this  example  the  optimal  stop¬ 
ping  set  must  be  of  the  form  =  [(t,  f]  for  some  cr  €  [0,  ^].  Conditions  which  determine 
the  optimal  value  <t  of  o-  depend  upon  the  signs  of  a  and  e. 


CASE  1;  a  >  0,  €  >  0.  Here  <f  +e  €  and  s(tf  +€)=  g(<T  -Lc).  Applying  Theorem 
I  and  taking  Equation  (10)  into  account,  we  find  that  if(f  €  (0,  f ),  then  it  must  satisfy 

(11)  2a(r -L \I(o-  -L  t)^  —  cr^]  —  =  0, 

a  quadratic  in  tr  Only  the  root 

o- ^  =  [a  -L  X€  +•  Via  -L  Xe)^  -L  ftXe^)/ b 

can  be  <t  and  one  has  <r  =  min  (o-  +  ,  f ). 


CASE  2:  a  >  0.  «  <  0.  In  this  case  <r  +  e  €  ^  and  Theorem  1  yields 
(12)  2acr  -LXs(<T  -  1«1)  -  (X  -L  6)0-2  =  0 


or  s(o-  —  |e  1)  =  o-  ((X  +  6)a-  —  2ol/X 

as  an  equation  which  must  be  satisfied  by  <r  if  it  lies  in  (0,  ().  Since  s(o-  —  |e  |)  ^  ^(o-  — 
|c  I)  =  (tr  -  !«  1)2,  Equation  (12)  implies  that  tr  must  satisfy  6o-2  -  2(a  -  X  |e  |)o-  -  X62  ^  0, 
or  (T  >  [a  -  X  kl  +  VTcT-TiriV^+^Xp)/ 6.  To  determine  <f  from  Equation  (12),  we  note 
that  solutions  of  the  functional  equation  4/(x)  =  bf(x),  x  €  (0,  o-);  fix)  =  0,  x  <  0; 
/(a)  =  0-2,  /'(o-)  =  2o-,  will  have  a  discontinuity  of  magnitude  /o  =  fi0+)  >  0  at  x  =  0, 
which  is  determined  to  satisfy  the  boundary  condition  f  i<r)=~  /(o-.  /o)“<z-2.  The  optimal 
value  of  (T  is  then  chosen  to  fulfill  f'i<T)=  2tr  or  /(o-  —  |e|)  — o-[(X  +  b)<T  —  2a]/X  (cf. 
Equation  (12)). 

CASE  3:  a  <  0,  «  <  0.  Here  x,  is  decreasing  with  probability  one  and  we  clearly  have 
.9^-  E and  six)  “  x^  for  all  x  €  f. 

CASE  4;  a  <  0,  €  >  0.  Theorem  2  applies  to  yield  s(<t-)  =  (f  2  as  the  optimality  condi¬ 
tion  forcr.  Since  x  =•  0  is  reflecting,  j'(0)  —  0,  or  s(*-L)  =  (1  +  6/X)s(0-L).  A  procedure  for 
determining  a-  is  then  the  following:  for  o-  €  (0,  f),  solve  /4/(x)  —  6/(x),  x  €  (0,  o-), 
fix)  ~  x2,  X  €  l(r.  ^1,  fix)  =^,  X  >  (.  Solutions  of  this  equation  will  generally  have  a 
discontinuity  of  magnitude  f^  =  fitr—)  at  x  -  cr,  which  must  be  chosen  for  this  <r  to  satisfy 
the  reflecting  boundary  condition  at  x  “  0.  Finally,  &  is  chosen  such  that  the  optimality  condi¬ 
tion  fi<T-)  -  <t2  is  also  met.  Note  that  the  resulting  function  fix)  (-  s(x))  will  then  be  con¬ 
tinuous  on  (0.  ^ )  and  will  also  have  a  continuous  derivative  there. 
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6.  CONCLUDING  REMARKS 

i.  If  a(x)  =  0  in  (0,  f )  or  A  (jr )  is  not  bounded  away  from  zero,  x,  may  not  attain  a  stop¬ 
ping  set  with  probability  one,  and  stopping  policies  must  generally  include  prescriptions  of  what 
actions  to  lake  if  .v,  enters  a  set  in  E  which  is  absorbing. 

ii.  Cost  functionals  other  than  (3)  may  sometimes  be  treated  within  the  present  frame¬ 
work.  The  costs 

Jo  ~  Jo 

and 

[a  -h  £o  Jo  c(jfs)</sj/£oT 

and  their  discounted  versions  are  such  functionals.  For  example,  in  the  second  case,  if 
P(t  <  oo )  =  1,  then  an  equivalent  problem  is  to  find  an  optimal  stopping  time  for  the  cost 

(13)  /(x) -h  £,F(Xr). 

where  f(x)  =  a’(x)  -  /(x)  and  /(x)  satisfies  /4/(x)  -  c(x)  =  0  (cf.  Taylor  [10]).  Nonauto- 
nomous  costs  or  dynamics,  or  both,  may  also  be  treated  by  augmenting  x,  with  time  as  a  second 
dimension. 

iii.  If  |e  I  «  1,  machine  solution  of  the  retarded  or  advanced  differential  equation 

4/'(x)  -  h/'(x)  =  0  may  cause  numerical  difficulties,  especially  if  there  are  singularities  caused 
by  zeros  of  a(x).  Singular  perturbation  expansions  of /(x)  in  the  form  ^  where#  is 

i>0 

a  suitable  stretching  transformation  (e  g.,#  =  x/e),  and  the  Fj  are  to  be  found,  may  be  used  in 
this  case  to  derive  accurate  approximations  to  s(x)  and 
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ABSTRACT 

Consider  a  network  G(N.  A)  with  n  nodes,  where  node  I  designates_its 
source  node  and  node  n  designates  its  sink  node.  The  cuts  (Z,.  Z/). 

;  -  1 . n  -  \  are  called  one-node  cuts  if  1  €  Z^.  n9  Z,.  Zi  -  111, 

Z,  C  Ziy.1  and  Z,  and  Z^.,  differ  by  only  one  node.  It  is  shown  that  these 
one-node  cuts  decompose  G  into  I  ^  m  $  n/2  subnetworks  with  known 
minimal  cuts.  Under  certain  circumstances,  the  proposed  one-node  decomposi¬ 
tion  can  produce  a  minimal  cut  for  G  in  O(n’)  machine  operations.  It  is  also 
shown  that,  under  certain  conditions,  one-node  cuts  produce  no  decomposition. 

An  alternative  procedure  is  also  introduced  to  overcome  this  situation.  It  is 
shown  that  this  alternative  procedure  has  the  computational  complexity  of 
O(n’). 

1.  INTRODUCTION 

Determining  the  location  of  minimal  cuts  in  a  flow  network  is  gaining  wider  importance  in 
recent  literature.  Jarvis  (8,  9]  shows  that  in  a  communication  network,  if  each  arc  is  assigned  a 
value  which  represents  the  effort  of  disabling  that  communication  link,  then  the  optimal  stra¬ 
tegy  to  disable  the  communication  between  the  message  sender  and  the  message  receiver  is  to 
destroy  the  links  (arcs)  on  a  minimal  cut.  Phillips  and  Dessouky  [16]  indicate  that  the 
time/cost  tradeoff  problem  can  be  converted  into  a  problem  of  maximal  flow  where  at  each 
iteration  the  activities  on  the  minimal  cut  are  crashed.  Rhys  [18]  and  Pickard  (17)  indicate  that 
a  selection  problem  of  shared  fixed  costs  can  be  converted  into  a  maximal  flow  problem  where 
minimum  cut  identifies  the  optimum  options  selection. 

Some  effort  has  been  made  to  find  a  minimal  cut  or  all  the  minimal  cuts  in  a  directed  net¬ 
work,  regardless  of  maximal  flow  (see  (7,  IS]).  However,  all  the  labeling  algorithms  for  finding 
the  maximal  flow  provide  the  location  of  a  minimal  cut  as  a  by-product  at  the  end  of  the  algo¬ 
rithms  (3,  4,  6l  The  maximal  flow  algorithms  proposed  by  Dinic  [2]  and  Karzanov  [12],  find 
the  maximal  flow  but  they  do  not  provide  a  minimal  cut.  However,  finding  the  location  of  a 
minimal  cut  in  these  algorithms  can  easily  be  achieved  by  applying  a  labeling  procedure  on  the 
network  once  the  maximal  flow  is  obtained. 

Among  the  efficient  labeling  algorithms,  the  first-labeled  first-scanned  algorithm  proposed 
by  Edmonds  and  Karp  [3]  has  a  theoretical  upper  bound  of  (n^  -  n)/4  augmentations,  where  n 
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is  the  number  of  nodes  in  the  flow  network.  Among  the  efficient  nonlabeling  algorithms, 
Karzanov's  [12]  algorithm  with  preflows  has  the  theoretical  upper  bound  of  O(n^)  machine 
operations. 

In  this  paper  we  propose  two  decomposition  algorithms  for  locating  a  minimal  cut  in  a 
given  flow  network.  The  network  is  decomposed  into  1  <  m  <  y  subnetworks  with  known 

minimal  cuts  by  utilizing  a  one-node  cut  concept.  Section  2  gives  the  decomposition  of  the  net¬ 
work  into  two  subnetworks  by  an  arbitrary  cut.  The  relationship  between  the  minimal  cuts  of 
the  subnetworks  and  the  arbitrary  cut  is  given  in  Theorem  1.  The  concept  of  one-node  cuts 
and  some  related  theorems  are  introduced  in  Section  3.  Section  4  presents  an  algorithm  for 
decomposing  the  original  network  into  three  subnetworks  where  the  intermediate  network  con¬ 
tains  a  minimal  cut.  Finally,  conclusions  are  given  in  Section  S. 

2.  MINIMAL  CUTS  AND  NETWORK  DECOMPOSITION 


A  network  G(N.  A)  is  a  collection  of  nodes  and  ordered  pairs  of  nodes  which  are  called 
arcs.  N  designates  the  node  set  and  A  designates  the  arc  set  of  the  network. 


There  are  two  special  nodes  in  an  n-node  flow  network;  one  is  called  the  source  node, 
denoted  by  1  (or  s),  and  the  other  one  is  called  the  sink  node  and  is  denoted  by  n  (or  t). 


With  every  arc  (/,  j)  €  of  G  we  associate  a  positive  integer  Ujj,  called  the  capacity  of 
the  arc.  A  set  of  nonnegative  integers  fa  is  called  a  feasible  flow  in  a  network  if  they  satisfy 
the  following  equations; 


fij  fjk 


-/  if  J-i 
0  it  j  ^  1.  n 
f  if  J~  n 


and 

0  ^  fij  ^  Ujj  for  all  (/.  j)  €  A, 

where  /  is  a  nonnegative  integer  called  the  value  of  the  flow.  A  set  of  feasible  flow,  ffj,  which 
maximizes  /  is  called  a  maximal  flow. 


_  DEFINITION:  Let  AT  be  a  subset  of  the  nodes  in  G  such  that_I  €  A",  n  ?  AT.  Also  let 
X  N  -  X.  Let  {X.  X)_  be  a  set  such  that  the  arc  (/,  j)  €  (Af,  X)  if  i  ^  X,  J  ^  X  and 
(/.  J)  €  A  or  7  €  Af,  I  €  AC  and  (i.  j)  €  A.  Such  a  set  is  called  a  cut  (or  a  cut-set). 

The  capacity  of  a  cut  (AC.  X),  denoted  by  c(AC,  ^),  is  given  by 
c(AC.  A)-  X  w„. 

a  x.ik  x 


DEFINITION:  A  cut  with  minimum  capacity  in  a  network  is  called  a  minimal  cut  (min- 

cut). 
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DEFINITIOI^:  A  cut  (X,  X)  is  said  to  lie  on  the  left  of  a  cut  (K  ?)  if  -IT  C  Y.  Con¬ 
versely,  a  cut  ( Y,  Y)  is  said  to  lie  on  the  right  of  a  cut  (AT.  Af)  if  K  2  AT. 

DEFINITION:  A  cut  (AC.  AC)  is  said  to  lie  between  two  cuts  (Y.  Y)  and  (Z.  Z)  with 
K  C  Z,  if  K  C  AT  £  Z. 

DEFINITION:  Two_cuts  {X,  Af^andJT,  Y)  are  said  to  cross  each  other  if  each  one  of 
the  sets  Af  n  Y,  X  r\  T,  AT  n  Y,  and  Af  n  T  is  nonempty. 

Ford  and  Fulkerson  [4]  show  that  for  single  commodity  network  flow  problems  the  value 
of  a  maximal  flow,  /*,  is  equal  to  the  value  of  the  capacity  of  any  minimal  cut  in  that  network. 

Let  G  —  (yv.  /4)  a  network  with  A/  —  {1 . n).  Also,  let  (Af,  Al”)  be  an  arbitrary 

cut.  Let  all  the  arcs  (i.  X)  €  (X,  X)  be  collected  to  an  artificial  node  s  with 

U/j  —  ^  Ujj  for  each  /  €  AC. 

iiX 

Similarly,  let  the  arcs  (AC.  J)  €  (AC,  AC)  emanate  from  s  with 
Ujy  -  X  “i/  fo*"  j  ^ 

i€X 

Note  that  the  arcs  (J,  ')  ^  (AC.  AC)  are  ignored  in  this  procedure  (since  they  have  no  effect  on 
cut  values). 

By  this  procedure  we  create  two  subnetworks  connected  to  each  other  via  a  single  artificial 
node  s.  Let  Gi  =  (A'  U  {s},  Ai)  be  the  first  subnetwork  where  the  arc  set  Ai  contains  all  the 
arcs  (/,  J)  €  A  and  i  ^  X,  J  ^  X  plus  the  arcs  (/,  s),  i  €  AC  as  described  above.  Similarly,  let 
G2“_(|5)  U_AC,  A2)  be  the  other  subnetwork,  where  A2  contains  all  the  arcs  (/.  J)  6  A  and 
i  ^  X^J  ^  X  as  well  as  the  new  arcs  is,  J),  y  €  AC  as  described  above.  Note  that  c(AC,  s)  - 
cis,  AC)  =  c(AC,  X).  Also  note  that  the  artificial  node  s  serves  as  a  sink  node  for  Gj  and  as  a 
source  node  for  G2. 

THEOREM  1:  Let^a  network  GiN,  A)  be  partitioned  into  two  subnetworks  G\  and  G2 
by  an  arbiftary  cut  (AC,  AC),  as  described  above.  Suppose  that  (Zj,  Z|)  is  a  minimal  cut  for  G| 
and  (Zy.  Z2)  is  a  minimal  cut  for  G2.  Then  the  following  statements  are  true: 

(a)  max{c(Z,.  Z,),  c(Z2.  Z2)}  <  c(AC.  X), 

(b)  there  exists  a  minimal  cut  for  G  which  lies  between  (Z|,  Z])  and  (Z2,  Z2), 

(c)  if  c(Zi,  Z\)  -  c(AC,  AC)  >  c(Z2,  Z2),  then  (Z2,  Z2)  is  a  minimal  cut  for  G, 

(d)  if  c(Z2,  Z2)  -  c(AC.  X)  >  c(Zi.  Zj),  then  (Z|,  Z])  is  a  minimal  cut  for  G, 

(e)  if  c(Z,.  Z,)  -  c(Z2.  Z2)  -  c(AC.  X),  then  the  cuts  (Z,.  Z,),  (Z2.  Z2)  and  (AC.  X) 

are  all  minimal  cuts  for  G. 
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(a)  If  c(Zi,  Z^)  >  ciX,  Jf),  thenjve  would  have  chosen  X)  as  the  minimal  cut 
between  I  and  s.  Thus,  c(Z|.  Z\)  <  c(X,  X).  With  the  same  line  of  argument  we 
have  c(Zi.  Z^)  <  c{X.  X). 


(b)  Suppose  that  {Y.  Y)  is  a  minimal  cut  for  G  which  crosses  (Z],  Z\)  and  (Zj.  Zj)  as 
shown  in  Figure  1  below. 


o 


(X.  X)  (Zj,  Z2)  (Y,  Y) 

Figure!,  a  minimal  cut  (  K  F)  crossing  (Z,.  Z|)  and  (Zj.  Z}). 

Consider  the  following  node-sets; 

^  -  ?_n  z,  n  Zj,  B_-  K  n  z,  n  Z2,  c  -  ?  n  z,  n  Z2,  D  =  r  n  z,  n  Zz.  £- 
r  n  Z,  n  Z2,  f-  K  n  Z,  n  Z^^  Note  that  a  U  B  -  Zi,  CUDU£Uf-Zi 
.4U£UCUD-Z2,  £Uf-Z2,  BUDUf  -  Fand  A  V  C  U  E  -  Y. 

Consider  the  cut  (£.  N  -  B).  Since  B  c  Zi  this  cut  lies  completely  in  Cj  and  thus 
c(B.  N-  B):^  ciZx.  Z,) 

where  N-B’“AUCUDUEUF.  Thus,  we  have 

(1)  c(B.  A)  +  c{B,  C)  +  c(B.  D)  -E  c(B.  £)  +  c{B,  F)  >  dA.  C)  +  c{A,  D) 

+  c(A.  E)  +  c(A.  F)  -F  c(B.  C)  +  c(B.  D)  +  c(B.  £)  +  c{B.  F). 

Similarly,  consider  the  cuts  {N  —  E,  £)  and  (Z2,  Z2)  which  lie  in  the  subnetwork  Gj. 
With  the  same  line  of  argument  we  have 

(2)  c(A.  E)  +  c(B.  £)  +  c(C.  £)  -F  c(D.  £)  -F  c(F.  £)  >  c(A.  E)  -F  c(B.  £) 

-F  c(C,  £)  +  c(D,  £)  -F  c(A.  F)  +  c(B.  F)  +  c(C.  F)  +  c(D.  F). 
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Adding  (1)  and  (2)  together  and  subtracting  c(i9,  £),  ciC.  E),  c{B.  D),  c(A,  E)  and 
ciB,  F)  from  both  sides  we  get 

(3)  c(B,  A)  +  ciB,  C)  +  ciB.  E)  +  c{D.  E)  +  c{F.  E)  >  c(A.  C)  +  c(A.  D) 

+  c(A.  E)  +  2c(A.  F)  +  c(B.  C)  +  c(B,  E)  +  c(B.  F)  +  c{D.  E) 

+  c(C.  F)  +  c(D.  F). 

We  also  have 

(4)  0  > -c(/4.  D)  -  c(/4.  F)  -  c(C  F), 

(5)  c(D.C)-c(D.C) 
and 

(6)  ciD.  A)  +  c(F.  A)  +  c{F,  C)  >  0. 

By  adding  (3),  (4),  (5)  and  (6)  we  get 

(7)  c(B.  A)  +  c(B.  C)  +  ciB.  E)  +  c{D,  E)  +  c(F,  £)  +  c(0.  C)  +  cW.  A)'' 

+  c(F,  A)  +  c(F.  C)  >  c(A,  C)  +  c(A.  E)  4-  c(A.  F)  +  c(B.  C) 

+  c{B.  E)  +  c(B.  F)  +  ciD,  C)  +  c(D.  E)  +  c(D,  F). 

The  left-hand  side  of  (7)  is  the  capacity  of  the  minimal  cut  ( Y,  Y)  and  the  right-hand  side  is 
the  capacity  of  the  cut  (/4  U  B  U  D,  C  U  E  V  F).  Thus, 

(8)  c(Y.  Y)  ^  ciA  \J  B  U  D.  C  U  E  U  F). 

The  inequality  in  (8)  can  only  be  satisfied  as  an  equality.  Therefore,  we  conclude  that  the  cut 
(A  U  B  U  D,  C  V  E  U  F)  is  also  a  minimal  cut  for  G. 

(c)  Since  c(Zi,  Z])  -  c(X.  X)  both  are  minimal  cuts  for  Gj.  From  the  inequality  (8) 
in  part  (b)  we  have  shown  that  there  exists  a  minimal  cut  for  G  which  does  not  cross 
a  givei^  minimal  cut  for  0^.  Since  (X,  X)  is  a  minimal  cut  for  Gy  and  since 
clZj.  Z2)  <  c(X.  X)  then  (Zj.  Z2)  is  a  minimal  cut  for  G. 

(d)  With  the  samejine  of  argument  as  in  part  (c),  we  have  (X.  X)  as  a  minimal  cut  for 
G2  and  c(Zi,  Zj)  <  c(X,  X).  Therefore,  (Z|,  Zj)  is  a  minimal  cut  for  G. 

(e)  Since  cfZj.  Zj)-  c(X,  X)  —  ciZ^,  Z2),  by  considering  (X,  A')  as  a  minimal  cut 
for  G|  as  in  p^t  (c),  we  conclude  t^t  we  can_hnd  a  minimal  cut  for  G  which  does 
not  cross  (A',  A’).  Therefore,  (Z|,  Z\),  {Z^,  Zi)  and  (A’,  X)  are  all  minimal  cuts 
for  G. 

Consider  subnetworks  G  i  and  Gi-  Since  both  are  legitimate  networks  with  a  source  and  a 
sink  node,  they  can  similarly  be  partitioned  into  two  subnetworks  Gn,  G12,  and  G21,  G^, 
respectjyely,  by  arbitrary  cuts  (Ai,  A|)  and  (A2,  A2)  where  Ai  C  A  and  A  C  A2.  JLet 
(Z||.  Z||)  and  (Z12,  Z12)  be  minimal  cuts  for  Gu  and  G12,  and  let  (Z21.  Z21)  and  (Z22>  Z22) 
be  minimal  cuts  for  G21  and  G22>  respectively.  This  situation  is  depicted  in  the  figure  given 
below  where  sy  is  the  super  node  between  Gn  and  G12,  52  is  the  super  node  between  G21  and 
G22  and  5  is  the  super  node  between  G|  and  G2. 
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Figure  2.  6|  and  are  further  decomposed  into  (7||.  fri2.  and  C21.  622- 


Therefore,  we  can  hope_to  find  a  minimal  cut  (Z],  Zy)  for  Gi  by  considering  (A'l,  X\) 
and  the  minimal  cuts  (Zu.  Zn)  and  (Zjj,  Z12)  for  subnetworks  G\\  and  G12,  respectively. 
This  process  leads  us  to  the  idea  of  ultimate  decomposition  which  we  will  call  one-node  decom¬ 
position.  If  this  partitioning  of  subnetworks  is  repeated  iteratively,  X,  and  Xj^i  will  eventually 
differ  from  each  other  by  one  node.  The  following  section  introduces  the  one-node  cuts  and 
some  important  property. 

3.  ONE-NODE  CUTS 

Let  a  network  G(N,  A)  contain  n  nodes.  Also,  let  the  nodes  of  the  network  be  num¬ 
bered  from  1  to  n,  where  node  1  is  the  source  node  and  node  n  is  the  sink  node. 

Consider  the  following  noncrossing  cuts: 

(Z,.  Z,).  —  1,  2 . (n  ~  i).  where 

z,-u;.,0i.  z,~N-Zi. 


Thus, 


Z,-  {!).  Z2-  {1.  2) . Z,_,-  (1.  2 . n-  ll, 

Z,-  {2.  3 . nl.  Z2-  (3.  4 . n) . Z,_,-{nl. 


We  will  call  ^ese  cuts  one-node  cuts  since  Z,  differs  from  Zj+i  by  one  node.  The  values  of  the 
cuts,  c(Zj,  Z,),  /-  1 . n-  1  viewed  as  a  sequence  of  numbers,  will  produce  subse¬ 

quences  of  increasing  and/or  decreasing  values. 

Graphically,  we  will  have  a  picture  similar  to  the  one  given  in  Figure  3  below.  In  this  pic¬ 
ture  the  downward  arrows  represent  the  decreasing  subsequences  and  upward  arrows  represent 
the  increasing  subsequences. 
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FltiURE  3.  A  typical  picture  of  one-node  cuts  and  their  relative  values. 


THEOREM  2;  If  ciZ^.  >  c(Zi+,,  Z,+,)  for  /  -  1.  2 . n  -  1,  then  (Z„_,.  Z,_,) 

is  a  minimal  cut  for  G. 

PROOF:  Let  ( Y,  K)  be  a  minimal  cut  for  G  which  lies  on  the  left  of  (Z„-\,  Z„_|).  Con¬ 
sider  the  node  set  Y.  We  can  find  an  index  k  such  that  Zj  £  Y  for  J  ^  k  and  Zj  i^Y  for 
j  >  k.  (Note  that  Z\  Q  Y.)  Let  Figure  4  depict  this  situation. 


<2a.  2a» 


Figure  4.  The  case  for  r(Z,.  Z,)  >  c(Z,+).  Z,+\),  i  -  1.  2 . n  -  2. 
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Let  Z*+,  n  K  -  A/,  Z*+,  n  y  -  U  +  1}  -  N,  Z*+i  n  y  -  L  and  Z*+,  n  Y  -  T. 

We  have 

2(k)  ^  ^*+l) 

or 

c(W,  JVU  LU  r)^c(MU  JV.  LU  T) 
or 

c(Af,  ^)  +  c{M.  L)  +  o(W.  n  >  c(A/.  L)  +  c(W.  T)  +  c(M  L)  +  c(M  D. 

Since 

0  <  c(N.  L) 

we  can  write 

(9)  c(W.  /V)  +  c(A/.  T)  >  c(/V,  n  +  ciM,  T). 

Since 

(10)  ciL.T)-ca.T) 
and 

(11)  c(L,  iV)  >  0. 

by  adding  (9),  (10),  and  (11),  we  get 

(12)  cU.  T)  +  c(L,  N)-^c(M.  T}-h  c(M.  N)>  c(N,  T)  +  c{M.  D  +  cU.  T). 

The  left-hand  side  of  (12)  is  nothing  but  c(l',  K)  and  the  right-hand  side  is  c(y  U  {*  +  1), 

Y  —  [k  +  1|).  Therefore, 

(13)  c(n  ?)>  c(y  U  U -i-l).  T D). 

Hence,  the  cut  (T  U  {/c  -t-  1),  Y  —  [k  -Y  1})  is  as  good  a  cut  as  O',  Y).  By  assuming  the  new 
cut(yu  Y  —  {k  -<■  II)  as  a  minimal  cut  for  G  we  can  apply  the  same  reasoning  again. 

Thus,  we  will  eventually  reach  the  conclusion  that  (Z,-|.  Z„-i)  is  a  minimal  cut  for  G. 

THEOREM  3.  If  c(Z,.  Z,)  <  c(Z,+,,  Z,+,),  i  -  1 . (n  -  2),  then  (Z,,  Z,)  is  a 

minimal  cut  for  G. 

PROOF;  The  proof  of  this  case  is  very  similar  to  that  of  Theorem  2.  In  this  case  we  find 
an  index  k  such  that  Zy  3  Y  for  J  k  and  Zj  'i  Y  for  J  <  k.  This  situation  is  depicted  in  Fig¬ 
ure  S  below. 

We  define  M  -  Z*_,  n  y,  ^  -  Z*_,  n  Y,  L  -  Z*_,  n  y,  and  f-  Z*_,  n  ?. 

We  have 

c(Z*_i,  Z*_i)  <  c(Zt,  Z*) 
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(Y.  Y) 

o 
o 

M 


or 

c{M  U  L.  ^  U  n  <  ciM  KJ  L\J  N,  T) 
or 

ciM.  N)  +  ciM.  T)  +  c{L.  N)  +  c{L.  T)  ^  c{M.  T)  +  c(t,  D  +  c{N,  T). 

Since  c(I,.  N)  >  0, 

we  have 

(14)  c(M.  N)  +  ciM.  T)  <  c(Af.  T)  +  c(^,  T) 
and 

(15)  c(M.L)-c(M.L) 
and 

(16)  Q^c(^.L). 

By  adding  (14),  (15),  and  (16),  we  get 

c(W.  N)  +  c(M.  T)  +  c{M.  L)  ^  c(M,  T)  +  c{N,  T)  +  c{M,  L)  +  c(N,  L) 
or 

ciY-  Ul,  r  u  [k])  <  civ.  ?). 

As  in  Theorem  2  by  rede"nin|_the  minimal  cut  O'.  ?)  and  applying  the  same  argument,  we 
reach  the  conclusion  that  t  ?|,  Z|)  is  a  minimal  cut  for  G. 
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THEOREM  4:  If  the  one-node  cuts  produce  only  a  decreasing  sequence  followed 
immediately  by  an  increasing  sequence,  then  the  one-node  cut  with  the  smallest  capacity  is  a 
minimal  cut  for  G. 

PROOF:  Let  p  be  the  index  where  c(Z,.  Z,)  >  f(Z,+|^Z,+|)  for  1  ^  <  (p  -  1)  and 

(■(Z,,  Zj)  <  c(Z,+|,  Z,+i)  for  p  <  /  <  (n  -  2).  Let  (Zp.  Zp)_divide  the_network  into  two 
subnetworks  G\  and  G2  as  described  before.  (Note  that  (Zp,  Zp)  is  (X,  X)  in  Theorem  1.) 
The  subnetwork  Gi  contains  only  a  decreasing  sequence  and  the  subnetwork  G^  contains  only 
an  increasing  sequence.  From  Theorem  2,  (Zp,  Zp)  is  a  minimal  cut  for  G\  and  from  Theorem 
3,  (Zp,  Zp)  is  a  minimal  cut  for  G2.  Then  from  Theorem  1  we  conclude  that  (Zp,  Zp)  is  a 
minimal  cut  for  G. 

THEOREM  5;  If  the  one-node  cuts  produce  only  an  increasing  sequence  followed  by  a 
decreasing  sequence,  then  either  the  one-node  cut  with  minimal  capacity  is  a  minimal  cut  for 
G,  or  else  there  exists  a  minimal  cut  for  G  which  crosses  the  one-node  cut  with  the  largest  cap- 
city. 


PROOJ^;  Let  p  be  an  index  such  that  c(Z,,  Zj)  <  c(Z,+|.  Zi+\)  for  1  <  /  <  (p  —  1) 
and  c(Z,,  Z,)  >  c(Z,+|,  Z,+|)  for  p  <  /  <  (n  -  2).  Let  (Zp,  Zp)  decompose  G  into  two 
subnetworks  G]  and  Gj  ^s  described  before.  Rom  Theorem  3  we  get  (Z^,  Z\)  as  a  minimal 
cut  for  G|  and  from  Theorem  2  we  get  (Z„_|,  Z„_|)  as  a  minimal  cut  for  C2.  Therefore,  from 
Theorem  1-b  we  conclude  that  there  exists  a  minimal  cm  for  G  whichjies  between  (Zj,  Z,) 
and  (Zp_|,  Zp_|).  If  this  minimal  cut  is  not  either  (Z,,  Z))  or  (Z„^,  Zp_|),  then  it  cannot  lie 
completely  in  one  subnetwork.  Hence,  it  must  cross  the  cut  (Zp,  Zp)  which  separates  G\  and 
G2. 

COROLLARY  1:  For  any  network  G(N,  ^),  the  one-node  cuts  decompose  G  into 
I  <  m  <  (n/2)  subnetworks  with  known  minimal  cuts.  These  subnetworks  are  characterized 
as  the  collection  of  nodes  such  that  the  one-node  cuts  corresponding  to  these  nodes  form  a 
subsequence  of  consecutive  decreasing  and  increasing  values.  If  the  first  subsequence  is  an 
increasing  one  then  the  nodes  corresponding  to  this  subsequence  constitute  the  first  subnet¬ 
work.  If  the  first  subsequence  is  a  decreasing  one  then  the  nodes  corresponding  to  the  first 
decreasing  and  increasing  subsequences  constitute  the  first  subnetwork.  After  the  first  subnet¬ 
work  Gj  is  determined,  the  remaining  subnetworks  G/,  /  -  2 . m  can  be  established  by 

considering  the  nodes  in  each  consecutive  decreasing  and  increasing  subsequences.  If  the  one- 
node  cuts  produce  a  single  sequence  of  increasing  or  decreasing  values,  then  a  minimal  cut  for 
G  can  be  found  by  choosing  the  smallest  capacity  one-node  cut. 

A  minimal  cut  for  each  subnetwork  is  given  by  the  one-node  cut  in  each  subnetwork  with 
minimal  capacity.  (Note  that  the  last  subnetwork  may  contain  only  a  decreasing  sequence.) 

The  computational  complexity  of  decomposition  by  one-node  cuts  can  easily  be  given  as 
(n  ~  1)^  additions  and/or  subtractions  and  (n  -  2)  comparisons. 

Therefore,  the  one-node  cuts  can  produce  a  minimal  cut  for  a  network  in  0(/)^)  additions 
and/or  subtractions  if  the  one-node  cuts  produce  a  single  subnetwork  (cases  in  Theorems  2,  3, 
and  4). 
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It  is  clearly  obvious  that  it  is  to  our  benefit  to  end  up  with  a  single  subnetwork  (the  net¬ 
work  itself)  as  the  result  of  one-node  cuts.  The  following  section  exploits  this  situation.  In  the 
approach  presented  in  the  next  section,  the  one-node  cuts  are  selected  in  such  a  way  that  either 
the  one-node  cuts  will  produce  a  single  subnetwork  or  we  hope  that  some  nodes  will  be  elim¬ 
inated  from  the  original  network  so  that  a  smaller  subnetwork  can  be  solved  for  a  minimal  cut. 

4.  THREE-DECOMPOSITION  PROCEDURE 

The  name  three-decomposition  stems  from  the  fact  that  we  would  like  to  decompose  a 
given  network  G  into  three  subnetworks  G),  G2,  and  Gj  with  the  property  that  a  minimal  cut 
for  G  can  be  found  by  only  considering  the  intermediate  subnetwork  G3. 

In  order  to  achieve  this  goal  we  will  try  to  obtain  a  decreasing  subsequence  of  one-node 
cuts  starting  from  node  1.  If  this  process  does  not  exhaust  all  the  nodes  of  G  then  we  will  try 
to  obtain  an  increasing  subsequence  of  one-node  cuts  ending  at  node  n.  This  process  will  pro¬ 
duce  one  of  the  following  three  outcomes; 


(a) 

c(Z/.  Z,)  >  c(Z,+,.  Z,+i) , 

/■-I . n  -  2 

(b) 

c(Z,.  Zj)  <  c(Z,+|.  Z,+i)  , 

/-  1.  2 . n  -  2 

(c) 

c(^i,  2^,)  ^  Zi^\) , 

i  —  I . (k  -  1 )  and 

c(Z,_,,  Z,_|)  <  c(Z,.  Zt). 

i  “  (n  —  1),  (w  —  2) . P  +  1 

and  k  ^  p. 


If  the  outcome  is  as  given  in  case  (a)  then  from  Theorem  2,  Z^\)  is  a  minimal  cut  for 

G.  If  the  outcome  is  as  given  in  case  (b)  then  from  Theorem  3,  (Zj,  Zj)  is  a  minimal  cut  for 
G.  We  have  three  possible  subcases  for  an  outcome  described  in  case  (c): 

(i)  k  “  p.  In  this  subcase  the  network  contains  a  decreasing  and  an  increasing  block 
and,  hence,  (Z^^.  Z^)  is  a  minimal  cut  for  G. 

(ii)  2  <  p_-  /c  <  3.  In^  this  case  (Z*.  Z*)  is  a  minimal  cut  ^r  G  if  c(Z*,_Z*)  < 
c(Zp.  or  (Zp,  Zp)  is  a  minimal  cut  for  G  if  c(Zp,  Zp)  <  c(Z*,  Z*).  If 
c(Zj,.  Z*)  =  c(Zp,  Zp),  then  both  cuts  are  minimal  cuts  for  G. 

(iii)  p  -  k  ^  4.  In  this  case  either  the  one-node  cut  with  minimal  capacity  is  a  minimal 
cut  for  G  or  else  there  exists  a  minimal  cut  which  lies  between  (Z*,  Z*)  and 
(Zp.  Zp). 

The  following  algorithm  formalizes  the  foregoing  discussion. 

ALGORITHM. 

Step  0.  Let  Z,  -  {!),  Z,  =  -  (I}.  Also  let  c(Z,,  Z,)  »  T  «i;,  and  k  =  1. 

i\jnA 
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Step  1.  Determine 


c(2*  U  !;*),  2*  -  U*))  -  min  lr(2*  U  (>}.  2*  -  [j])]. 


K  c(2*,  2J  >  c(Z^  U  [j*\,  2*-  ly*!),  then  set  2*+,-  2;^  U  {j*\  and 

2*+i  =  N  —  2*+i,  “  /c  +  1,  go  to  Step  1.  Otherwise,  go  to  Step  2. 

Step  2.  Let  2,..,  =  {n}.  2„_,  =  -  2„_,.  Also  let 

c(2,_i,  2„_,) «  ^  u,„.  andp“w-l. 

Step  3.  If  p  =  4:,  go  to  Step  4;  otherwise  determine 

f(2,  -  {>•).  2^  u  {7*))  =  min  U(Z^  -  I7),  Z,  U  \j\)]. 


Step  4. 


If  f(2,.  2,)  >  r(2,  -  {yl,  2,  uO*!),  then  let  2,_,  =  2,  -  {y),  2,_,  = 

/V  and  p  —  p  —  I,  go  to  Step  3.  Otherwise,  go  to  Step  4. 

Step  4.  If  2*j-  2p  stop,_a  minimal  cut  for  the  original  network  G(N.  A)  is  at  hand. 
(2*,  2*)  “  (Zf,,  2p)  is  a  minimal  cut,  with  capacity  c(2*.  2*)  already  calculated. 

If  p  -  /c  <  3  (i.e.,  2k  and  2^  differ  by  at  most  3  nodes)  then  (2*.  f*)  is  a 

minimal  cut  for  the  original  network  if_c(2k.  2*)  <  c(2,.  2^)  or  (2^.  2^)  is  a 

minimal  cut  for  G  if  c(2^,  2^)  <  ciZ^,  2^),  or  else  both  cuts  are  minimal  cuts  for 
G. 

If  p  ~  k  >  3,  then  let  (2*.  2*)  and  (2^,  2^)  divide  the  node_set  of  the  original  net¬ 
work  into  three  subsets  A,  =•  2*,  Aj=  2^-  2*,  and  A3  -  2^,  respectively.  Con¬ 
struct  the  intermediate  subnetwork  G2(AJ,  /^j),  where  AJ-  Aj  U  (s*,  f*),  where  s* 
and  r*  represent  a  super  source  and  a  super  sink,  respectively.  Determine  Wj.y  and 
Wye.  7  €  A2  as  described  before.  Obtain  the  maximal  flow  for  G2  by  using  Karzanov's 
algorithm  [12].  At  maximal  flow  apply  Ford  and  Fulkerson’s  [4]  labeling  algorithm 
once  to  obtain  a  minimal  cut.  The  minimal  cut  obtained  by  this  procedure  is  a 
minimal  cut  for  the  original  network,  G. 

Assuming  Karzanov’s  (12)  algorithm  requires  O(n^)  operations,  we  can  determine  the 
computational  upper  bound  for  the  proposed  algorithm.  Let  us  assume  that  after  one-node 
decomposition  we  have  |2*1  -  k,  l2j  ~  n  -  p,  also  let  x  -  k  +  (n  -  p).  Thus,  the  toul 
number  of  additions  and/or  subtractions  for  Steps  0  through  3  can  be  written  as  O(xn^).  Simi¬ 
larly,  the  number  of  comparisons  can  be  written  as  Olxn).  If  k  »  1  and  p  -  n  -  l(x  -  2), 
then  ^e  algorithm  requires  O(n^)  additions  and  0(n)  comparisons  and  concludes  that  either 
(2i,  2|)  or  (2„_i.  2„_,)  (whichever  has  the  smallest  capacity)  is  a  minimal  cut  for  G  or  else  a 
minimal  cut  should  be  determined  on  the  overall  network  by  utilizing  any  efflcient  maximal 
flow  algorithm  available. 

If  2  <  X  <  (n  -  4)  thenjhe  algorithm_will  produce  an  intermediate  network  G2  which 
will  lie  between  the  cuts  (2*,  2*)  and  (2^,  2^).  Including  the  super  source  s*  and  the  super 
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sink  /*,  G2  will  contain  n  -  x  +  2  nodes.  Finding  the  maximal  flow  in  this  network  will  give 
us  a  computational  upper  bound  of  0[(ft  -  x  -F  2)^]  in  Step  4,  if  Karzanov’s  (12]  algorithm  is 
used.  Assuming  that  the  upper  bound  is  reached  in  Karzanov’s  algorithm,  then  the  computa¬ 
tional  upper  bound  of  the  proposed  decomposition  algorithm  can  be  written  as  0((r  —  x)^). 

5.  CONCLUSIONS 

Two  alternative  decomposition  procedures  have  been  presented.  In  the  first  procedure, 
one-node  cuts  were  selected  in  a  fixed  predetermined  order.  If  we  naively  assume  that  the 
values  of  one-node  cuts  are  random  and  coming  from  the  same  distribution,  then  we  can  find 
out  the  expected  numbers  of  runs,  E(X),  (increasing  or  decreasing  blocks)  from  the  formula 
flA-)-  (2(jV-  D-  l)/3-  (2^-3)/3  and  its  variance  F(3r)  -  IlblA' -  1) - 

291/90-  ilSN  —  45)/90  given  by  Levene  114],  where  (/V  —  1)  is  the  total  number  of  observa¬ 
tions  (one-node  cut  values).  For  (A^  —  1)  >  20,  the  distribution  of  X  can  be  approximated  by 
a  normal  distribution  with  mean  E(X)  and  variance  ViX)  as  given  above  [14]. 

Therefore,  in  a  large-scale  network,  the  probability  of  having  X  ^  2  can  be  obtained  from 
the  statistic 

Z  -  ”  <-2^^  +  9)/[(16)V  -  45)/10]''^ 

For  N  -  22,  PiX  <  2)  -  <t>  (-6.317)  =  1.0  x  10"’. 

The  probability  obtained  above  indicates  that  the  probability  of  having  one  or  two  blocks 
in  (N  -  1)  cuts  is  very  small.  In  other  words,  the  probability  of  hitting  the  worst  case  is  almost 
zero  (if  the  two  blocks  are  an  increasing  and  a  decreasing  one)  as  well  as  the  probability  of 
finding  the  minimal  cut  without  further  computation  (if  there  is  either  one  block,  or  two  blocks 
starting  with  a  decreasing  one).  However,  if  the  arc  capacities  are  identically,  uniformly  and 
independently  distributed  between  two  integers,  a  and  b,  and  if  the  network  is  complete,  then 

EiUfj)  n  •  (b  +  a)f  2 

and 

y{U/j)  -  a-fj"’  (b  —  o)Vl2. 

Thus,  the  expected  capacity  of  the  cut  (Z*,  Z*)  can  be  written  as 

£{c(Zfc,  Z*)}  —  k(n  —  k)n  —  k(R  —  k)  (6  +  o)/2 
and 

F{c(Z*.  Z^))-  [kin  -  k)]Hb- a)V  12. 

where  kin  -  k)  is  the  number  of  arcs  on  the  cut  (Z^.  Z^^)  from  the  nod^set  Z^  to  the  node 
set  Z|^.  It  is  clear  from  the  above  formulas  that  the  one-node  cut  (Z^/j.  Z^i^)  has  the  largest 
expected  capacity  and  also  the  largest  variance. 

We  have  simulated  some  random  networks  with  arc  capacities  uniformly  and  indepen¬ 
dently  distributed  over  the  range  (1,  50)  and  the  number  of  increasing  and/or  decreasing 
sequences  are  given  in  Table  I  below. 

It  is  clear  from  the  table  that  a  sparse  network  can  be  decomposed  into  more  subnetworks 
than  a  dense  network. 
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TABLE  1  —  Average  Number  of  Increasing  and/or 
Decreasing  Blocks  on  Random  Networks 


Number  of 
Nodes 

Number  of 
Replications 

Density  Ul/ 

n(n~ 

1  ) 

.1 

.2 

.3 

.4 

.5 

.6 

.7 

.8 

.9 

1.0 

10 

50 

3.30 

3.92 

1.82 

T36 

116 

3.02 

Tso 

2.64 

2.44 

2.24 

20 

50 

6.5 

5.96 

5.48 

4.48 

4.16 

3.64 

3.76 

3.40 

2.64 

2.80 

30 

50 

10.02 

7.88 

6.56 

5.72 

4.96 

4.76 

4.24 

3.76 

3.40 

3.00 

40 

50 

11.52 

8.88 

7.20 

6.32 

5.72 

4.52 

3.32 

4.32 

3.60 

3.16 

50 

50 

13.78 

10.04 

8.82 

6.88 

6.08 

5.36 

4.36 

4.32 

3.88 

3.72 

75 

50 

18.08 

12.6 

9.68 

8.20 

7.12 

6.08 

5.48 

4.84 

4.40 

4.20 
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ABSTRACT 

in  (his  paper  we  consider  dual  angular  and  angular  structured  mixed  integer 
programs  which  arise  in  some  practical  applications.  For  these  problems  we 
describe  efficient  methods  for  generating  a  desirable  set  of  Benders'  cuts  with 
which  one  may  initialize  the  partitioning  scheme  of  Benders.  Our  research  is 
motivated  by  the  computational  experience  of  McDaniel  and  Devine  who  have 
shown  that  the  set  of  Benders'  cuts  which  are  binding  at  the  optimum  to  the 
linear  relaxation  of  the  mixed  integer  program,  play  an  important  role  in  deter¬ 
mining  an  optimal  mixed  integer  solution.  As  incidental  results  in  our  develop¬ 
ment,  we  provide  some  useful  remarks  regarding  Benders'  and  Dantzig-Wolfe's 
decomposition  procedures.  The  computational  experience  reported  seems  to 
support  the  expedients  recommended  in  this  paper. 


1.  INTRODUCTION 

In  this  paper  we  consider  some  mixed  integer  programs  with  special  structures.  Our 
research  is  motivated  by  the  empirical  study  of  McDaniel  and  Devine  (10)  who  discovered  that 
when  solving  a  mixed  integer  program  using  the  decomposition  scheme  of  Benders  (2),  the  set 
of  Benders’  cuts  or  constraints  in  the  master  problem  which  are  binding  at  the  linear  program¬ 
ming  optimum  are  almost  sufficient  to  obtain  an  optimal  solution  to  the  mixed  integer  program, 
in  that  only  a  few  additional  Benders’  constraints  are  required.  In  fact,  working  with  problems 
of  different  structures,  McDaniel  and  Devine  [10]  found  that  if  they  were  to  solve  the  linear 
relaxation  of  the  mixed  integer  program  using  the  decomposition  scheme  of  Benders  (2),  that 
is,  solve  the  master  program  at  each  iteration  as  a  linear  program,  then  when  they  switched 
over  to  solving  each  master  program  as  an  integer  programming  problem  after  having  obtained 
the  linear  programming  optimum,  very  few  (typically  one)  additional  Benders’  inequalities  were 
required  to  verify  optimality. 

In  the  light  of  this  study,  we  examine  two  types  of  mixed  integer  programs,  each  with  a 
special  structure.  For  these  programs,  we  describe  a  means  of  exploiting  their  special  structures 
to  obtain  an  initial  set  of  Benders’  cuts  with  which  one  may  initialize  the  usual  partitioning 
scheme  of  Benders  (2).  This  initial  set  of  cuts  is  desirable  in  view  of  McDaniel  and  Devine’s 
(10)  observation,  in  that  the  linear  relaxation  of  the  master  program  with  these  cuts  obtains  an 
optimal  or  near  optimal  solution  to  the  linear  relaxation  of  the  original  mixed  integer  program. 
As  incidental  results  in  our  analysis,  we  provide  some  interesting  remarks  regarding  Berniers’ 
[2]  and  Dantzig-Wolfe's  [4]  decomposition  procedures. 
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2.  MIXED  INTEGER  PROGRAMS  WITH  DUAL  ANGULAR  STRUCTURE 

In  this  section,  we  consider  a  mixed  integer  program  of  the  form 

PI:  minimize  cj  x\  4-  cj  X2  +  d‘y 

subject  to  /4iXt  +  Dfy  > 

A 2X2+  D2y  '>  *2 
Gy  >  g 

JCi.  X2.  y  >  0 
y  €  n 

where  n  is  the  set  of  integer  vectors  and  where  a  superscript  t  will  throughout  denote  a  matrix 
transpose  operation.  Problem  PI  is  said  to  have  a  dual  angular  structure  with  the  variables  y 
being  coupling  variables  (see  [7]  for  example).  For  the  sake  of  noutional  simplicity,  we  have 
considered  only  two  diagonal  blocks  of  x-variables,  though  of  course,  the  development  general¬ 
izes  to  any  number  of  such  blocks. 

Problem  PI  may  arise,  for  example,  within  the  context  of  discrete  stochastic  programs 
with  recourse  [13]  where  in  addition,  the  "first  stage  decision*  vector  y  is  restricted  to  be  integer 
valued  (typically  binary- valued).  The  proposed  approach  in  this  case  becomes  particularly 
attractive  if  the  random  right-hand  side  vector  b  can  only  be  one  of  a  reasonably  few  number  of 
vectors  b^,  b2 . b,  according  to  some  discrete  probability  distribution. 

Now,  let  LPl  denote  the  linear  relaxation  of  Problem  PI,  that  is.  Problem  PI  with  the 
restrictions  €  fl  relaxed.  Further,  merely  for  the  sake  of  simplicity,  assume  that  each  set  of 
variables  xj  and  X2  has  one  component  which  is  an  artificial  variable  with  a  large  positive 
coefficient  in  the  objective  function  and  a  column  of  ones  in  its  corresponding  constraint- 
coefficient  matrix.  This  assumption  guarantees  that  a  feasible  solution  to  Problem  LPI  exists 
for  any  fixed  value  of  >’  €  Y,  where 

(1)  F-  (>-;  Gy>g.  y^  0l 

Using  the  partitioning  scheme  of  Benders  l2].  Problem  LPl  may  be  decomposed  as 
min  d'y  +  [min  c'lX]  +  C2X2 :  A^x^"^  b\—  D\y,  X)  >  0 

.v«  r  x,.*j 

A2X2  >  b2  -  Diy,  X2  >  0) 


which  may  be  rewritten  (using  duality)  as 

(2)  min  d'y  +  [{max  (bi  —  DiyYiri  :  A\ir^  <  Cj,  Wi  >  0) 

+  [max  (^2  ~  f>2>’)'»’2’  A'2ir2  4  C2.  "’2  >  0)1 

Finally,  denoting 

(3)  5)  -  ItT)  : <  Cl.  W|  >  0) 

(4)  Sj-  [it 2  :  A'2it2  <  Cj.  It 2  >  0) 
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and  observing  that  our  assumptions  imply  that  5|  and  S2  are  bounded.  Problem  LPl  may  be 
written  as  Benders’  master  problem 


BMP: 

minimize 

subject  to 

d'y  -P  zi  +  Z2 

(5) 

Zi  >  (/»!  —  Diy)%  ^  for  each  J  6  Jj 

(6) 

where 

Z2  >  (b2  —  D2y)^2  ^  ^  "^2 

y  e  Y 

(7) 

J\  "  [j  'lr  ( is  an  extreme  point  of  S| ) 

(8) 

•f 2  “  ly :  ^  is  an  extreme  point  of  S2 ) 

Let  this  problem  yield  an  optimal  vector  y*.  Then,  the  corresponding  xj  and  xi  which  define 
an  optimal  solution  (xt,  xf,  y*)  to  Problem  LPI  are  obtained  respectively  from  (2)  as  dual 
optimal  vectors  to  Problems  SPl  (y*)  and  SP2  (y*),  where 

SPl(y);  max  l«»i  -  ir,  6  Si) 

SP2Cy):  max  Hb2-  DjyYvi  :  itj  €  Sj). 

However,  not  all  the  constraints  in  (S)  and  (6)  may  be  necessary  to  define  an  optimal 
solution  (zf,  zj,  y*)  to  Problem  BMP.  As  shown  by  Benders  [21,  the  following  relaxation  of 
BMP  may  be  sufficient  so  long  as,  if  (z|.  Z2.y)  solves  BMP  (k,  /),  then  zi  and  £2  are  respec¬ 
tively  the  optimal  objective  values  of  Problems  SPl  (y)  and  SP2  (y); 

BMP(/c./):  minimize  d'y  +  Z|  -t-  Z2 

subject  to 

(9)  z,  >  (*|  -  Z)|y)‘irj'  for  J-  ....  k 

(10)  Z2  >  ib2~  D2y)^{  fory  “  1 . I 

y  €  Y. 

If  this  is  not  the  case,  say,  zj  <  (bi  -  DiylV*"'^’,  where  solves  SPI  (y),  then  an  addi¬ 
tional  Benders’  cut  (or  constraint)  zi  >  (fr|  -  D^y)'  nf'^'  is  appended  to  Problem  BMP  (k,  /). 
In  this  manner,  Benders’  121  scheme  iterates  between  the  master  problem  BMP  (k,  /)  and  the 
subproblems  SPl  (y)  and  SP2  (y)  until  for  some  k  and  /,  the  above  termination  criterion  is 
satisfied.  It  is  our  purpose  to  show  how  one  may  construct  such  a  problem  BMP  (k,  /)  is  a 
more  efficient  manner.  Thereafter,  as  indicated  by  McDaniel  and  Devine  (lOl,  if  one  com¬ 
mences  the  application  of  Benders’  scheme  on  Problem  PI  with  the  master  program 
BMP(it.  /),  then  a  minimal  amount  of  additional  cuts  of  the  type  (9),  (10)  may  be  required  to 
solve  PI. 

The  main  property  we  use  toward  this  end  is  the  dual  relationship  between  Benders’  [21 
scheme  and  the  Dantzig- Wolfe  [4)  decomposition  method  (see  Lasdon  [7],  for  example). 
Observe  that  the  dual  (LDI)  to  Problem  LPI  has  a  block  diagonal  (or  angular)  structure  as 
shown  below 
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LDl:  maximize  b\nx  +  +  gin 

subject  to 

D'l  IT  I  +  DJir  j  +  Gix  ^  d 
ir,  €  S, 

ff  j  €  Sj 

a  ^  0. 

This  problem  is  most  amenable  to  the  Dantzig-Wolfe  decomposition  method  [4]  which  writes 
each  TT ,  €  Si  as  a  convex  combination  ir  i  —  ^^xjir  /,  where  ^  X  |y  —  1,  X >  0,  and  simi- 

>€/  JfJx 

larly  expresses  each  vj  €  Sj.  In  this  manner.  Problem  LDl  is  written  as  the  Dantzig-Wolfe 
master  program. 

DWMP:  maximize  ^XiyfAiW^-f  ^  Xjy  -1- a'g 

iiJx  JiJi 

subject  to 

1  X,;  (D'xwp  +  £  -t-  GW  ^  d 

X  X  -  1  ,  £  ^  2y  “  1 

JiJ\  JiJi 

Xiy  ^  0,  y  €  di;  X2/  ^  0,  y  €  /j;  a  ^  0. 

Again,  not  all  the  columns  for  Xiy,  J  €  J\  and  X22,  y  €  Jj  may  be  necessary  to  solve  LDl 
through  DWMP.  In  particular,  the  following  restriction  of  DWMP  may  be  sufficient. 

k  I 

DWMP(X,  /);  maximize  +a'g 

y- 1  y- 1 

subject  to 


(11) 

^X,y(D'|ir^)  +  ^X2y  (DJtt p -1-  Cfa(  <  d 
/-I  y-i 

(12) 

y-i 

(13) 

K2J  ”  li  Xi^  ^  0,  y  “  1,  . . .  ,  Ac;  ^  0>  y  “  1>  •  •  • 

y-i 

,  / ;  a  >  0 

so  long  as,  if  Z|,  and  22  are  optimal  dual  variables  associated  with  constraints  (11),  (12)  and 
(13),  respectively,  then  zi  and  Z2  respectively  the  optimal  solution  values  of  Problems 
SPl  (p)  and  SP2  (p).  If  this  is  not  the  case,  say  f|  <  (bj  -  D^yV  iri*'*'',  where  solves 
SPl  (p),  then  an  additional  column  corresponding  to  iri*'*''  (with  variable  Xk^^+d)  is  appended 
to  Problem  DWMP  (.k,  I).  In  this  manner,  Dantzig- Wolfe’s  I4l  method  iterates  between  the 
master  problem  DWMP  (k,  I)  and  the  subproblems  SPl  (y)  and  SP2  (y)  until  for  some  A:  and 
I,  the  above  termination  criterion  is  satisfied. 
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Note  the  dual  nature  of  the  foregoing  remarks  [7],  In  fact.  Problems  DWMP  and  BMP 
are  duals  of  each  other  and  so  are  Problems  DWMP  (k,  /)  and  BMP  (k,  1).  Thus,  once  we 
have  solved  LDl  and  terminated  with  some  DWMP  (k,  /),  one  can  easily  write  out  the  Prob¬ 
lem  BMP  (k,  I)  which  solves  LPl.  The  reason  why  it's  preferable  to  solve  LDI  is  of  course 
the  savings  in  effort  whereby  the  Dantzig-Wolfe  decomposition  scheme  maintains  a  basis  of  a 
fixed  size  as  opposed  to  one  of  an  ever  increasing  size  when  using  Benders'  scheme.  This  may 
be  more  pronounced  if  >'  is  a  vector  of  a  few  variables  and  Y  has  several  constraints  and  less 
pronounced  otherwise. 

We  will  consider  next  a  mixed  integer  program  with  an  angular  structure.  Although  this 
structure  does  not  lend  itself  to  as  straightforward  an  analysis  as  above,  we  are  nonetheless  able 
to  make  some  pertinent  observations  and  remarks  which  are  supported  by  our  computational 
experience. 

3.  MIXED  INTEGER  PROGRAMS  WITH  ANGULAR  STRUCTURE 

Consider  a  mixed  integer  program  P2  with  the  following  angular  structure. 


P2: 

minimize 

c'x  +  d'y 

subject  to 

(14) 

Cx  +  Dy  >  f 

(15) 

Ax  >  b 

Gy  >  g 

X  >  0.  ^  >  0  and  y  €  ft 

where  ft  is  the  set  of  integer  vectors.  Problem  P2  may  be  used  to  model  fixed  charge  produc¬ 
tion  and  location,  and  other  related  problems,  where  the  vector  y  is  constrained  to  be  binary 
valued,  denoting  a  yes/no  type  of  decision,  and  the  vector  x  represents  production  or  allocation 
variables.  As  before,  let  LP2  be  the  linear  relaxation  of  P2,  that  is,  with  the  constraint  >>  €  ft 
relaxed,  let  Kbe  as  defined  in  Equation  (1),  and  assume  that  the  set 

(17)  X  •  \x:  Ax  ^  b.  X  >  0} 

is  nonempty.  For  simplicity,  we  assume  that  both  X  and  Y  are  bounded  in  the  following 
development.  Also,  we  have  considered  just  one  block  of  constraints  in  (15),  although  it  may 
be  thatx'-  (x',,  ....  x,'),  c'-  (cj . c/),  (b\  ....  b',),  C-  (C, . C,l,  and 

/I,  0 

A  - 

0  ■  A, 

Again,  we  will  assume  that  the  vector  x  has  an  artificial  variable  as  one  of  its  components,  with 
a  large  positive  objective  function  coefficient,  and  a  column  of  ones  in  the  rows  (14)  and  zeroes 
in  the  rows  (15).  This  will  then  guarantee  that  for  any  fixed  >'€)',  there  exists  a  vector  x  such 
that  (x.  >*)  is  feasible  to  LP2. 
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Now,  one  may  partition  Problem  LP2  using  Benders'  decomposition  [2]  scheme  as  fol¬ 
lows; 

min  d'y  {min  c'x  :  Cx  ^  f—  Dy,  >  fe,  x  >  0) 

>€  r  X 

=  min  d'y  4-  (max  (/  -  DyY  it  +  b'a:  C'lr  -f-  /f'a  <  c,  ir  ^  0,  a  >  0). 

Y  If. a 

Defining  the  sets 

(18)  S  =  {(it,  a) ;  C'n  +  A 'a  ^  c,  it  ^  0,  a  ^  0) 

(19)  y  -  {j ;  (rr^,  are  extreme  points  of  5} 

a  relaxed  Benders’  master  problem  with  k  Benders’  cuts  may  be  written  as 


MP(A:): 


minimize  z 


subject  to 


z  d'y  +  (J  -  Dy)'n'’  -(-  frla'’  for  each  p  ^  K  Q  J 
y  e  Y 


where  . k\.  Then  if  (z.  y)  solves  Problem  MP(Ar),  one  would  solve  the  subprob¬ 

lem  SP  {>)  or  its  dual  SD  (p),  where 


SPCv); 


and  its  dual  is 


SDfj-): 


maximize  [d'y  +  (/-  DyY  it  -4  fefat;  (it.  a)  €  S) 


minimize  [d'y  +  c'x:  Cx  f-  Dy,  Ax  ^  b,  x  ^  0). 


If  it  turns  out  that  z  “  d'y  +  c'x  where  x  solves  SD  (p),  then  (x,  >)  solves  LP2.  Otherwise, 
z  <  d'y  +  c'x,  and  a  Benders'  cut  z  ^  d'y  +  if  —  Dyl'ir**'  -4  ft'a*'*'’  is  appended  to  rows 
(20),  where  (rr**'.  a*^')  solves  SP(y).  The  process  now  continues  in  this  manner. 

Note  that  every  time  a  new  row  is  added  to  (20),  a  solution  to  Problem  SD(y)  is  called 
for,  a  problem  with  an  angular  structure.  However,  Problem  LP2  itself  has  just  one  more  diag¬ 
onal  block  than  Problem  SDfy),  Therefore,  the  application  of  Dantzig-Wolfe’s  [41  decomposi¬ 
tion  to  the  former  problem  has  a  basis  of  dimension  one  more  than  that  applied  to  the  latter 
problem.  Moreover,  while  using  Dantzig-Woife’s  decomposition,  we  would  be  able  to  exploit 
any  special  structures  which  the  set  Y  may  have.  This  advantage  may  be  lost  in  the  solution  of 
problems  MP(Ar)  for  \K\  ^  2.  Hence,  we  propose  to  do  the  following.  We  will  solve  Problem 
LP2  using  Dantzig-Wolfe’s  decomposition  method  {4).  From  this  solution  process,  we  will 
show  how  one  may  obtain  a  desirable  set  of  cuts  (20)  so  as  to  avoid  expending  effort  in  gen¬ 
erating  cuts  which  may  not  be  binding  at  the  linear  programming  optimum.  From  this  point 
onwards,  one  may  continue  with  the  solution  of  Problem  LP2  using  Benders'  decomposition 
and  then  switch  over  to  solve  Problem  P2  as  advocated  by  McDaniel  and  Devine  [101.  In  con¬ 
clusion,  we  will  also  make  a  remark  which  may  be  a  further  expedient  in  the  latter  process. 
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First  of  all,  let  us  present  a  pertinent  observation  afforded  by  Balas  and  Bergthaller  [I]. 
Note  that  if  (x*.  y*)  solves  LP2,  then  the  set  of  Benders'  cuts  which  are  binding  at  optimality 
are  indexed  by  the  following  subset  of  the  set  J  of  Equation  (19) 

(21)  7  -  O'  €  y :  (ir^,  is  optimal  to  Problem  SPCv*)). 

However,  not  all  the  cuts  indexed  by  j  €  7  may  be  necessary  to  determine  y*  in  the  Benders’ 
master  problem  at  optimality.  In  particular,  if  y*  has  S  positive  components,  then  SD(y*)  has  S 
degenerate  basu;  variables  at  optimality,  and  so  SPfy*)  has  up  to  2*  alternative  optimal  solu¬ 
tions.  Thus,  |y|  <  2®  with  the  equality  often  holding  II].  But  the  dimension  of  Benders'  mas¬ 
ter  program  is  n  -F  1 ,  where  y  has  n  components.  Further,  we  know  that  n  -  pof  the  hyper¬ 
planes  y,  >  0  /■  €  {1 . rt)  hold  as  an  equ^ity  at  y*.  Thus,  at  most,  some 

(n  +  1)  -  (n  ~  p)  =  p  +  I  Benders'  cuts  indexed  by  J  may  be  needed  to  define  the  linear  pro¬ 
gramming  optimum  y*.  We  now  proceed  to  demonstrate  how  one  may  generate  a  desirable 
subset  of  cuts  indexed  by  /  Our  discussion  will  motivate  the  choice  of  this  subset.  In  addi¬ 
tion,  we  will  show  how  some  more  Benders’  cuts  which  are  binding  at  optimality  may  be 
recovered  during  the  application  of  Dantzing-Wolfe’s  decomposition  procedure  to  Problem 
LP2,  without  any  extra  effort. 


Toward  this  end,  suppose  that  we  have  solved  Problem  LP2  using  Dantzig-Wolfe’s 
decomposition  method.  Let  the  Dantzig-Wolfe  master  program  at  optimality  be 


' 


DWMP2: 

minimize  £  (c'Xp)  +  £  y,  (d\) 

(22) 

p- 1  «- 1 

£  Xp  (Cx„)  +  £  y,  (Oy,)  >  f 

p-\  1 

(23) 

£  Xp  =>  1,  X^  >  0  for/>  -  1,  . 

p-i 

(24) 

£  y,  “  1 .  y,  ^  0  for  <7  -  1,  . 

Suppose  (X  *.  >  •)  solves  DWMP2  with  X*>  0  p  -  1 . k,  y*  >  0  q  1 . /.  Then 

(x*.  y*)  solves  LP2  where  x*  -  ^  k^{x^),  y*  -  £  yj(y,).  Here,  x,^  vert  X  (Equation 

1  <)- 1 

(17))  and  y,«  vert  Y  (Equation  (ID).  Further,  if  tt  *,  ft  *  and  •  are  optimal  dual  variables 
associated  with  the  constraints  (22),  (23)  and  (24)  respectively,  then  ft  *  and  ^  *  are  respectively 
the  optimal  solution  values  of  Problems  SPx(ir  •)  and  SPy(rr  •),  where 

SPjr(w):  minimize  |(c  -  CSr)'x  :  x  €  X] 

SPy(7r);  minimize  {{d  —  DhrVy  :  y  Y}. 

Lemma  I  below  recovers  a  dual  optimal  solution  to  Problem  LP2  from  the  Dantzig-Wolfe  mas¬ 
ter  and  subproblems  at  optimality.  This  result  may  be  viewed  as  an  extended  special  case  of 
the  more  general  result  due  to  Magnanti,  Shapiro,  and  Wagner  [81. 
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LEMMA  1:  Let  Problems  DWMP2,  SPx(ir)  and  SP^Cir)  and  the  vector  ir  •  be  as  defined 
above.  Further,  let  a  *  and  ^  *  be  optimal  dual  variables  in  Problems  SPx(ir  *)  and  SP^fir  •), 
respectively.  Then,  (n  *, «  *,  /3  *)  is  an  optimal  dual  solution  to  Problem  LP2. 

PROOF;  From  the  duals  to  Problems  SPx(ir*)  and  SPy(ir*),  it  follows  that 
(it  *  a*  fi  •)  is  dual  feasible  to  LP2.  Further,  with  x*  y*,  •  and  0  *  as  defined  above,  the 

Dantzig-Wolfe  termination  criterion  implies  that  P  *  -  Z>far  •  and  0  •  —  ^)3  *.  Thus,  from 
DWMP2,  c'x*  +  d'y*  -  /hr  •+»*+</>*•  /^  •  +  b'a  *  +  gjS  *.  This  completes  the  proof. 

Noting  that  (tr*.  a*)  €  5,  let  us  construct  the  following  Benders’  master  problem  with  a 
single  cut  constraint. 

MP{*):  minimize  z 

(25)  subject  to  z  d'y  ■¥  {J  —  Dy)'it  *  +  b'a* 

y  6  Y. 


Now,  consider  the  following  result.  In  view  of  Lemma  1,  this  result  is  closely  associated  with 
the  concept  of  "strongest  surrogate  constraints,"  a  name  given  to  constraints  of  type  (25).  (See 
GeofTrion  I5l,  Glover  (6l  and  Rardin  and  Unger  (111). 

LEMMA  2:  Let  (x*,  .v*)  and  in*,  a*.  13*)  be  optimal  primal  and  dual  solutions  respec¬ 
tively  to  Problem  LP2.  Then  y*  is  optimal  to  MP(*)  with  z*  =  c'x*  +  d'y*. 

PROOF;  A  Lagrangian  dual  to  Problem  LP2  is 

LD;  maximize  [hizr,  a)  :  v  ^  0,  a  0} 

where  hin.  a)  —  min  [d'y  -*-(/—  Dy)HT  +  b'a]  +  min  {(c  -  Chr  —  /4fat)'x}. 

►  x^O 


Since  there  is  no  duality  gap  between  LP2  and  LD,  we  know  that  (w  *,  a  *)  solves  LD  with 
objective  value  h{zr*.  a*)  equal  to  c'x*  +  d'y*.  Further,  (x*.  y*)  evaluates  hiir*.  a*).  But 
by  complementary  slackness  in  LP2,  the  second  problem  in  h(n  *,  a  *)  has  a  value  of  zero  at 
X  —  X*.  Thus,  the  first  problem  in  bin*,  a  *),  which  is  Problem  MP(*),  has  an  optimal  solu¬ 
tion  y*  and  an  optimal  value  of  c'x*  +  d'y*.  This  completes  the  proof. 

Let  us  digress  momentarily  to  discuss  a  certain  issue  related  to  the  strongest  surrogate 
constraint  (25).  Often,  such  a  constraint  is  prescribed  as  a  device  for  somehow  fathoming  par¬ 
tial  solutions,  or  selecting  branching  variables  in  a  branch  and  bound  or  implicit  enumeration 
context,  implemented  within  the  framework  of  an  algorithm  which  solves  Problem  P2  using 
Benders’  decomposition.  Lemma  3  below  addresses  the  case  when  such  a  strategy  may  not  be 
very  meaningful,  that  is,  when  (25)  degenerates  to  simply  z  >  z*. 

LEMMA3;  Let  (x*.  y*)  and  (ir  *.  a*,  *),  respectively,  be  optimal  primal  and  dual  solu¬ 

tions  (of  value  z*)  for  Problem  LP2.  Then,  if  y*  €  int  V',  the  strongest  surrogate  constraint 
(25)  degenerates  to  r  >  z*. 
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PROOF:  If  y*  €  int  Y,  then/3  •  =  0  and  y*  >  0.  Thus,  z*  ”  fir  *  +  bin*,  and  the  dual 
constraints  Dhr  +  GjS  <  d  are  binding  at  (ir .  /3 )  -  (ir  *,  /3*).  In  other  words,  Dht  *  d. 

Noting  that  (25)  is  z  ^  {fit  *  +  *)  +  (</  —  Dht  *)'>’,  the  proof  is  complete. 

COROLLARY:  In  case  (25)  does  not  degenerate  to  z  >  z*,  then  y*  and  hence  each 

I 

v,  €  vert  1^,  where  >»•  —  ^y*q  Yq-.  belong  to  the  face  of  Y  defined  by  its  intersection  with  the 

q-l 

hyperplane  id  -  D'lr  *)'y  =  g'p  *. 

The  above  result  holds  trivially  from  Lemma  2  and  Equation  (25).  Its  significance  is  the 
subject  of  our  next  discussion. 

Now,  Lemma  2  seems  to  indicate  that  a  very  desirable  starting  point  at  which  to  com¬ 
mence  the  solution  of  Problem  LP2  by  Benders’  decomposition  is  with  Problem  MP(*).  In 

addition  to  this  single  cut,  one  may  also  choose  to  generate  the  following  desirable  set  of 
independent  cuts  which  will  be  binding  at  the  linear  programming  optimum,  i.e.,  which  are 
indexed  by  the  set  J. 

The  reader  may  note  from  the  corollary  to  Lemma  3  that  each  y^.  9  =  1 . /  is  an 

alternative  optimal  solution  to  MP(*).  Therefore,  it  is  intuitively  appealing  to  generate  / 

Benders’  cuts  in  addition,  one  for  each  y^  through  Problem  SPIj-,)  or  SD(y^)  for 1 . /. 

However,  these  cuts  may  not  be  binding  at  the  linear  programming  optimum,  that  is,  they  may 
not  be  indexed  by  the  set  J.  To  overcome  this,  consider  the  following  subproblem  which 
differs  from  SPCv,)  by  the  additional  constraint  (26) 

SP'(>',):  maximize  d'y^  -f  (/-  Dv,)5r  -F  b'a 

subject  to  (tt  ,  a )  €  S 

(/-  Dy*y  It  -F  Afar  >  z*  -  d'y*  —  c'x*. 

Lemma  4  below  characterizes  a  Benders’  cut  obtained  through  Problem  SP’Cv,).  Thereafter,  we 
show  how  to  solve  SP'Cv,)  while  still  exploiting  the  angular  structure  which  SEXir,)  possesses. 

LEMMA  4:  Consider  any  j',,,  ^  €  |I . /)  and  suppose  that  (tt’.  a")  solves  Problem 

SP'Cp^).  Consider  the  Benders’  cut 

(27)  z  >  d'y  +  {/-  Dy)'n‘'  +  Afa». 

Then  the  cut  (27)  satisfies  q  ^  1.  Moreover,  when  this  cut  is  appended  to  Problem  MP(*),  it 
deletes  j',  provided  there  does  not  exist  an  x,  €  X  such  that  (x,.  y^)  solves  LP2. 

PROOF:  Since  (z*.  y*)  is  an  optimal  solution  to  MP(y)  where  J  is  defined  in  Equation 
(19),  it  follows  that 

(28)  z‘  >  d'y*  -F  (/  -  Dy*)hr  -F  b'a 

for  any  (ir,  a)  €  S.  Noting  Equations  (26)  and  (28),  we  may  assert  that  the  cut  (27)  is 
indexed  by  </  €  /  Now,  y^  is  an  alternative  optimal  solution  to  MP(*).  Thus,  if  (x,.  >',)  does 
not  represent  an  optimal  solution  to  LP2  for  some  x,  €  X,  then  there  must  exist  some  u  €  7 
such  that  z*  <  d'y^  -F  (/-  Dy^Y  n"  ■¥  b'a".  But  (it“,  a")  is  feasible  to  SP'Cy,).  Hence,  we 
must  have  z*  <  d'y^  -F  (/-  Ov,)*ir’  -F  Afar*.  This  completes  the  proof. 
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We  will  now  proceed  to  show  how  one  may  solve  SP'(>>,)  with  no  more  effort  than  that 
required  to  solve  SPCy,)  or  SDO",).  Observe  that  for  a  fixed  -  y*  in  LP2,  we  obtain 
2*  -  d'y*  —  c'x*  as  the  maximum  value  of  (/  -  Dy*)hr  +  over  (ir,  a)  €  S.  Noting  Equa¬ 
tion  (26),  Problem  SP'(y,)  may  be  viewed  as  a  preemptive  priority  multiobjective  program  over 
the  set  S,  with  an  objective  if  —  Dy*)'ir  +  b'a  having  first  priority  and  an  objective 
(/—  Dy^Vir  -E  b'a  having  a  lower  priority.  According  to  Sherali  and  Soyster  [12],  there  exists 
a  positive  scalar  |  *  such  that  for  any  ^  an  optimal  solution  to  the  following  problem  is 

also  optimal  to  SP'Cj',). 

d'y^  +  maximum  {((1  +  f)/-  Dy^  ~  iDy*Yn  +  (1  +^)6'a:(jr,  a)  €  S). 


The  dual  to  this  problem  is  the  following  angular  structured  linear  program. 

SD'Cj',,  ():  d'y^  minimum  {c'x:  Cx  >  (1  +^)/-  Dy^  -  tDy* 

Ax  >  (1  +Ob.  X  >  0). 

Note  that  finding  a  |  large  enough  (|  ^  ^  should  be  no  problem  since  it  is  easy  to  show  (12| 
that  (  *  in  our  case  is  simply  the  negative  of  the  optimal  dual  variable  associated  with  constraint 
(26).  Moreover,  we  know  that  we  have  chosen  ^  large  enough  after  solving  SD'(>’,,  ^ )  by  veri¬ 
fying  that  (26)  is  binding  at  the  optimal  solution  in",  a"). 

Summarizing,  one  may  first  solve  LP2  using  Dantzig-Wolfe's  decomposition  and  retrieve 
from  that  (through  Lemma  1)  the  strongest  surrogate  constraint  (25).  Then  the  master  pro¬ 
gram  MP(*)  with  this  single  cut  constraint  determines  the  optimal  value  of  LP2.  However, 

since  any  point  in  the  convex  hull  of  |>| . jk’/l  is  also  optimal  to  MP(*),  one  may  determine 

for  each  q  *■  1 . /a  Benders'  cut  which  will  be  binding  at  optimality  and  will  also  delete  y, 

by  solving  SD'{y^.  O  for  a  large  enough  value  of  (  as  indicated  above.  In  concluding  this  sec¬ 
tion,  we  provide  below  two  remarks.  The  first  of  these  pertains  to  an  alternative  strategy  for 
generating  an  initial  set  of  Benders'  cuts  and  the  second  remark  further  addresses  the  issue  of 
strongest  surrogate  constraints. 

REMARK  I;  This  remark  is  an  alternative  to  generating  an  initial  set  of  Benders'  cuts  via 
Problems  SD'Cy,.  ^).  Note  that  without  any  extra  effort,  during  the  solution  of  Problem  LP2 
by  Dantzig-Wolfe's  decomposition  method,  certain  Benders'  cuts  are  automatically  available. 
As  the  following  discussion  motivates,  these  cuts  are  quite  likely  to  be  binding  at  optimality. 
Suppose  we  execute  the  Dantzig-Wolfe  procedure  as  follows.  At  any  stage,  having  generated 

extreme  points  yi . A  of  in  the  master  program,  we  continue  to  iterate  between  the 

master  program  and  the  subproblem  SPxlrr)  only  until  no  more  extreme  points  of  X  need  to  be 
generated.  At  this  point  in  time,  the  current  solution  is  clearly  optimal  to  the  problem 


DW(v): 

minimize 

c'x  +  Z  r, 

(29) 

subject  to 

Cx  +  £  y,  (£)y,)  >  / 

(30) 

/4x  >  6 

(31) 

in-  j 

«-i 


X  >  0,  y,  >  0  for  q  —  1 . v. 
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Currently,  suppose  that  rr'',  0''  and  are  optimal  dual  variables  associated  with  the  constraints 
of  the  type  (22),  (23)  and  (24),  respectively,  and  that  a''  is  an  optimal  dual  solution  to  the  sub¬ 
problem  SPx(7r'').  Then  following  Lemma  1,  ir'',  a''  and  may  be  shown  to  be  optimal  dual 
variables  associated  with  constraints  (29),  (30)  and  (31),  respectively.  But  this  means  that  if 

V 

(y .  x)  solves  DW{\),  then  for  a  fixed  y  ”  X,  ^  “'■®»  respectively,  optimal 

_  _ 

solutions  to  SDiy)  and  SP(y).  Therefore,  a  Benders’  cut 

(32)  z>  d'y  +  (J-  Dy)' ir'' +  b'a'' 
is  readily  available. 

Hence,  if  we  execute  Dantzig-Wolfe’s  procedure  by  generating  as  many  extreme  points  of 
X  as  required  before  we  generate  an  additional  extreme  point  of  K,  then  for  each  optimal  com¬ 
bination  of  extreme  points  of  Y  which  have  been  generated  at  any  stage,  we  have  a  Benders' 
cut  (32)  available  from  the  current  dual  variables.  It  is  intuitive  that  these  cuts  are  desirable. 
Of  course,  at  termination,  (32)  is  precisely  (25).  Also,  one  may  finally  choose  to  keep  those 
cuts  (32)  which  turn  out  to  be  binding  at  the  linear  programming  optimum.  We  provide  some 
computational  experience  in  Section  5  regarding  this  remark. 

REMARK  2:  This  note  is  related  to  the  strongest  surrogate  constraint  (25)  and  to  our 
comments  associated  with  Lemma  3.  Observe  that  by  also  accomodating  the  constraints 
Gy  ^  g  into  the  objective  function  of  Problem  LD  in  the  proof  of  Lemma  2,  it  is  easy  to  show 
that  the  following  cut  may  be  used  in  lieu  of  (25)  in  Problem  MP(*). 

z  d'y  ■¥  (,/'  -  DyVit  •  +  &(»*-♦•(«-  CylJS  • 

or 

r  >  (fir*  +  b'a*  +  g'p*)  +  (d  -  D'lr*  -  G'fi'Vy. 

This  is  again  the  strongest  surrogate  constraint  of  Rardin  and  Unger  (11).  But  fir*_+  b'a*  -F 
g'P*  =  z*,  the  optimal  objective  value  of  Problem  LP2,  and  (d  -  D'lr*  -  G'P*)  -  d  >  0,  the 
reduced  cost  coefficient  vector  of  the  ^^variables  at  the  linear  programming  optimum.  Thus, 
the  above  cut  is 

(33)  r  >  z*  -F  d'y. 

Observe  that  if  y*  has  all  positive  components,  then  d  -  0,  whence  (33)  is  simply  z  ^  z*. 
Also,  it  is  easy  to  see  that  if  (25)  degenerates  to  z  ^  z*,  then  so  does  (33),  though  not  neces¬ 
sarily  vice  versa.  For  example,  g)3  •  0  is  sufficient  for  (25)  not  to  generate  to  z  >  z*, 
though  not  for  (33).  Moreover,  since  our  master  program  treats  the  constraint  set  y  €  Y  expli¬ 
citly,  we  find  (25)  more  appropriate. 

4.  ILLUSTRATIVE  EXAMPLE 

Consider  the  following  problem  adapted  from  Lasdon  [7]. 

P2:  minimize!- 2xi  -  Xj  -  yj  -  y2:  -  2xi  -  X2  -  yi  -  2y2  >  -40.  x  €  y  €  Y 

and  y  integer) 

X-  (x:-x,  >  -10.  -X2  >  -10.  -X, -X2  >  -15.  x,.  X2  >  0) 

Y  -  {y:  -y,  -  3>’2  >  “30.  -2>-|  -yi  >  -20.  y,.  y2  >  0). 
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Using  Dantzig-Wolfe’s  decomposition  to  solve  LP2,  we  first  generate  a  vector  y  by  solving  the 
problem,  minl-yi  -  y2:  y  C  Kl.  This  yields  y  -  (6,  8).  Continuing  with  the  usual  procedure 
but  generating  only  vertices  of  X  as  needed,  that  is,  solving  DW(v  -  1),  yields  x  -  (4,  10)  as  a 
convex  combination  of  the  vertices  (0,  10)  and  (5,  10)  of  X.  For  this  solution,  ir''-  1, 
Ait''  -  0  and  hence  the  cut  (32)  is 

(34)  z  >  y2-  40. 

The  next  vertex  of  K  generated  through  SPy (ir")  is  y  —  (10,  0).  Now,  v  -  2  and  the 
solution  to  DW(v-  2)  through  the  continued  generation  of  extreme  points  of  X  yields 

X-  (10.  5),  a  single  vertex  of  X,  and  y  —  -^(6.8)  +  -jy  (10,  0)  —  -^j.  At  this 

stage,  ir"  ”  y,  bit''  ••  — ^  and  hence  the  cut  (32)  is 

(35)  2  >  -  jy,  -  y  yj- 30. 

However,  no  more  vertices  of  Y  need  to  be  generated  and  so  x*  =  (10.  5)  and  y*  “ 
solves  LP2  with  optimal  value  2*  “  -  Thus,  (35)  is  actually  the  strongest  surrogate  con¬ 
straint  (25).  Note  that  according  to  Remark  2,  that  cut  (33)  is  simply  2  >  -  Further, 

we  obtain 

MP(*);  min  I2;  2  ^  -  y  yt  -  y y2  -  30.  y  €  I'). 

This  problem  has  (6,  8)  and  (10,  0)  as  alternative  optimal  extreme  point  solutions  with  a  value 
of  r*  -  — Note  that  the  incidental  cut  (34)  automatically  obtained  during  the  solution  of 

LP2  is  binding  at  the  linear  programming  optimum.  Indeed,  this  is  also  the  cut  which  would  be 
obtained  through  SD'(y,.  0)  with  y,  =  (6,  8).  The  reader  may  easily  verify  that  the  problem 

SD'O,.  ^ ).  for  y,  -  (10.  0)  and  for  any  1^0,  yields  an  optimal  dual  solution  (ir^  a")  with 

it’ -  0  and  A'a" -  25.  Hence,  corresponding  to  y„  -  (10.  0),  the  Benders’  cut  which  is  binding 
at  optimality  is 

(36)  2  >-yi  -  >'2  -  25. 

It  turns  out  here  that  the  cuts  (34)  and  (36)  imply  the  strongest  surrogate  constraint  (35).  This 

1  2 

may  be  verified  by  using  the  surrogate  multipliers  y  and  y  on  (34)  and  (36)  respectively. 

Thus,  the  master  program  MP  (.K)  with  the  cuts  (34)  and  (36)  yields  in  this  case  an  optimal 
solution  to  LP2  using  the  fewest  number  of  Benders’  cuts.  In  general  one  would  have  contin¬ 
ued  solving  LP2  with  the  Benders’  cuts  (34),  (35)  and  (36). 

Now,  continuing  with  the  solution  to  Problem  P2,  we  obtain  an  optimal  solution  to 
MP(A)  with  y  restricted  to  be  integer  valued  as  y  —  (8,  4)  with  2  —  -36.  Solving  the  subprob¬ 
lem  SD(y),  we  obtain  an  optimal  solution  x  -  (10.  4)  also  with  value  -  36.  Thus,  (x.  y) 
solves  P2  and  we  terminate  without  having  to  generate  any  further  Benders’  cuts.  Indeed,  such 
a  propensity  is  borne  out  by  the  computational  experience  of  McDaniel  and  Devine  (iOl. 


25  iO 
3  ’  3 
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5.  COMPUTATIONAL  EXPERIENCE 

In  this  section,  we  will  attempt  to  support  the  ideas  recommended  in  this  paper  through 
some  computational  experience.  Toward  this  end,  recall  the  principal  thrust  of  this  paper. 
McDaniel  and  Devine  (lOl  have  amply  demonstrated  that  in  solving  a  mixed  integer  program 
via  Benders’  [2]  procedure,  an  attractive  strategy  is  to  solve  its  linear  relaxation  first  via 
Benders’  partitioning  method,  and  use  the  resulting  cuts  as  an  initial  set  of  cuts  for  the  former 
problem.  Hence,  we  have  attempted  in  this  paper  to  demonstrate  how  one  may  solve  the  linear 
relaxation  via  Benders’  decomposition  method  more  efficiently,  for  some  specially  structured 
mixed  integer  programs. 

The  discussion  in  Section  2  clearly  demonstrates  the  value  of  the  suggestions  for  dual 
angular  structured  problems,  particularly  when  there  are  few  j>-variables  and  several  more  con¬ 
straints  in  Y.  On  the  other  hand,  the  suggestions  in  Section  3  need  computational  support. 
More  specifically,  one  needs  to  demonstrate  how  well  the  proposed  initial  set  of  Benders’  cuts 
represents  the  linear  programming  solution,  as  well  as  the  savings  in  effort  which  accrues  from 
their  use.  Furthermore,  the  comments  in  Remark  1  need  to  be  tested. 

Hence,  we  performed  the  following  brief  computational  experiment  on  an  IBM  370  Model 
158  computer  with  coding  in  Fortran.  We  generated  linear  programs  of  the  form  to  minimize 
c’x  -F  d'y,  subject  to  coupling  constraints  Cx  +  Dy  ^  /,  x  >  0,  .v>  >  0  and  block  constraints 
Gy  ^  g,  y  ^  0,  with  c  and  d  generated  uniformly  on  [-11,-1],  C  and  D  generated  uniformly 
on  [-50,  0],  / generated  uniformly  on  I— 251,  —1],  #  generated  uniformly  on  [—6,  —  1]  and  with 
G  having  components  0  or  - 1  with  an  expected  density  of  0.5.  All  generated  coefficients  were 
rounded  off  to  the  nearest  integer. 

Table  1  summarizes  the  results.  The  "Usual  Benders’  Method*  is  being  performed  by 
treating  the  set  K  -  {y:  Gy  ^  g,  y  >  0}  as  the  set  of  complicating  variables.  The  "Section  -3- 
MethocT  generates  an  initial  set  of  cuts  via  problems  SD'(  )  and  then  continues  as  usual, 
whereas  the  "Remark-l-Method"  employs  the  recommendations  embodied  in  Remark  I  of  Sec¬ 
tion  3  to  generate  an  initial  set  of  cuts. 

Essentially,  both  the  methods  of  Section  3  perform  better  than  the  usual  Benders' 
approach,  with  that  of  Remark  1  appearing  somewhat  superior.  However,  as  in  Problems  1  and 
2,  the  Section-3-Method  seemed  better  than  the  Remark-l-Method  in  some  other  larger  sized 
problems  we  attempted  in  which  NB  >>  NC.  The  reason  being  that  the  Remark-l-Method 
generates  cuts  from  SF(y)  where  y  is  a  combination  of  some  extreme  points  of  Y,  and  when¬ 
ever  the  Dantzig-Wolfe  algorithm  generates  and  drops  off  several  y-vectors  while  solving  LP2, 
the  Remark-l-Method  cuts  are  not  as  effective. 

In  conclusion,  we  reiterate  that  the  savings  in  effort  as  reflected  through  columns  /  would 
have  been  more  enhanced  if  Y  had  a  special  structure,  since  this  could  have  been  exploited  in 
solving  problems  SPy(-)  though  not  necessarily  in  solving  MP(A)  for  iA|  >  2. 

6.  SOME  CONCLUDING  REMARKS 

In  conclusion,  we  present  two  further  expeditious  remarks.  The  first  of  these  pertains  to 
iteratively  updating  solutions  to  the  subproblems  SD(y)  rather  than  resolving  each  subproblem 
independently.  Essentially,  this  remark  in  general  contributes  towards  performing  sensitivity 
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TABLE  1  —  Computational  Results 


Problem 

Number 

m 

n 

NC 

NB 

Usual-Benders 

Method 

_ 

■  '  ■  -  -  -  -  -  1 

Section-3-Method 

Remark- 1 -Method 

a 

b 

c 

d 

e 

t 

c 

d 

e 

f 

1 

B 

B 

30 

40 

“l2 

3 

"T" 

T" 

0.427 

T” 

~T' 

T" 

0.445 

10 

40 

5 

3 

4 

4 

4 

0.760 

4 

5 

4 

0.900 

40 

40 

30 

40 

6 

3 

4 

4 

4 

0.850 

4 

4 

4 

wmi 

30 

30 

10 

40 

6 

3 

4 

5 

5 

0.973 

5 

5 

4 

0.757 

30 

30 

30 

40 

9 

3 

4 

5 

5 

0.648 

4 

4 

4 

20 

20 

10 

40 

4 

3 

4 

4 

4 

0.768 

3 

3 

3 

50 

50 

40 

30 

8 

2 

3 

3 

3 

0.626 

4 

4 

2 

0.431 

8 

50 

50 

40 

10 

8 

3 

4 

4 

4 

0.835 

5 

5 

4 

mmi 

Mtm 

40 

40 

40 

30 

7 

2 

3 

3 

3 

0.537 

2 

2 

2 

0.326 

30 

30 

40 

10 

5 

3 

4 

5 

5 

1.379 

3 

3 

3 

0.621 

Bm 

30 

30 

40 

30 

5 

3 

4 

5 

5 

1.270 

3 

3 

3 

0.622 

20 

20 

40 

10 

10 

3 

n 

4 

4 

0.625 

3 

3 

3 

0.286 

50 

50 

40 

40 

8 

2 

B 

3 

3 

0.604 

4 

4 

2 

0.432 

40 

40 

40 

40 

7 

2 

11 

3 

3 

0.477 

2 

2 

2 

30 

30 

40 

40 

5 

3 

II 

5 

5 

1.190 

3 

3 

3 

20 

20 

40 

40 

10 

3 

B 

4 

4 

0.494 

3 

3 

3 

isil 

(i<  m  -  #  of  X  variables;  P  of  y  variables;  NC  —  #  of  coupling  constraints;  NB  "  #  of  block  constraints  (in  Y). 


(ii>  a  =  total  number  of  Benders'  cuts  required:  6  =  #  of  cuts  in  V  which  were  binding  at  optimality. 

(iii)  c  =  #  of  initial  cuts  (including  strongest  surrogate  constraint);  </  =  total  number  of  cuts  required;  r  =  #  of  cuts  in 
•<r  which  were  binding  at  optimality;  f  =  ratio  of  total  solution  time  to  the  total  solution  lime  for  the  Usual- 
Benders-Method. 


analysis  in  the  context  of  Dan tzig- Wolfe's  decomposition  procedure.  However,  the  question 
regarding  its  computational  efficiency  is  open  to  further  research.  The  second  remark  briefly 
addresses  the  issue  of  effectiveness  of  Benders'  partitioning  scheme. 

REMARK  A:  This  remark  concerns  the  iterative  solutions  of  the  subproblems  SD(y), 
both,  while  continuing  to  solve  LP2  as  well  as  while  subsequently  solving  P2  using  Benders' 
decomposition.  Suppose  that  we  have  solved  SD(y)  for  some  y  using  Dantzig-Wolfe's  decom¬ 
position.  (Here,  Ax  b  may  be  one  of  several  angular  blocks  of  constraints).  Let  if  and  9  be 
optimal  dual  multipliers  for  the  constraints  Cx  f  —  Dy  and  the  convexity  constraint  for  the 
extreme  points  of  X,  respectively.  Then,  on  replacing  y  by  some  y  for  the  subsequent  solution 
of  Problem  SDiy),  we  have  a  dual  feasible  but  primal  infeasible  Dantzig-Wolfe  tableau.  Let 
the  Ah  row  in  this  tableau  have  a  negative  right-hand-side  value  and  denote  the  first  m  columns 
of  the  Ah  row  of  the  current  basis  inverse  by  Bi,  where  C  has  m  rows.  Further,  let  the  element 
of  the  basis  inverse  in  the  Ah  row  and  the  same  column  as  the  row  of  the  convexity  constraint 
for  the  x-variables  Then,  it  follows  that  the  reduced  cost  coefficient  (c  -  CW)'x  -  0  is 

nonnegative  for  each  x  €  vert  X.  Now,  in  order  to  perform  a  dual  simplex  iteration  in  row  /, 
we  need  to  generate  a  column  corresponding  to  a  vertex  x  of  X  such  that  its  updated  value 
BiCx_+ilii  in  row  /  is  negative  and  yields  the  smallest  value  for  the  ratio  |(c-Cff)' 
X  -9}/  ~  {BjCx  +((<,)  among  all  such  columns.  To  accomplish  this,  consider  the  following 
linear  fractional  program. 
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LFP;  minimize 

Note  that  if  one  solves  this  problem  using  Chames  and  Cooper’s  [3]  method  by  letting 
M  -  (1/  -  {B(Cx  +  *(1)  >  0  and  v  -  ox,  then  at  optimality,  we  will  have  u  >  0  since  SD(y) 
is  feasible  by  our  original  assumption.  This  transformation  converts  LFP  to  the  linear  program 

minimize  |(f  —  C'iF)'  v  —  9u:  Av  —  bu  ^  0,  B/Cv  +  i|ifU  —  — 1,  «  >  0,  v  >  0). 

Thus,  instead  of  solving  each  subproblem  from  scratch  using  Dantzig- Wolfe's  decomposition, 
one  may  update  the  solutions  to  subsequent  subproblems  through  dual  simplex  iterations  as 
indicated  above.  Note  that  this  is  a  general  technique  which  one  may  adopt  to  perform  a  sensi¬ 
tivity  analysis  on  right- hand-side  perturbations  within  the  context  of  Dantzig- Wolfe's  decompo¬ 
sition  method. 

REMARK  B:  It  has  been  conjectured  that  the  "nearei'  the  solution  to  the  linear  relaxa¬ 
tion  of  a  mixed  integer  program  is  to  a  solution  of  the  orginal  mixed  integer  program,  the  fewer 
the  number  of  Benders'  cuts  that  will  be  required  to  solve  the  mixed  integer  program.  The  dis¬ 
cussion  is  this  paper  tends  to  support  this  statement  in  the  following  manner. 

Suppose  that  it  turns  out  that  LP2  has  an  optimal  solution  (x*.  >>*)  satisfying 
y*  €  K  n  fl .  Let  z*  -  f'x*  -t-  d'y*  and  let  (ir  *,  a  *.  /5  •)  be  a  dual  optimal  solution  to  Prob¬ 
lem  LP2.  Then,  the  strongest  surrogate  constraint  (25)  is  also  the  Benders'  cut  which  would  be 
generated  through  SP(>’*).  Hence,  using  Lemma  2,  it  follows  that  the  Benders’  master  problem 

minimize  z 

subject  to  z  >  dV  +  (/  -  Dy)'n  *  +  bb  * 

>  €  y  n  n 

with  the  single  cut  (25)  has  (z*.  >*)  as  an  optimal  solution.  Thus,  it  is  possible  for  the  pro¬ 
cedure  to  terminate  after  generating  a  single  Benders’  cut.  In  any  case,  if  one  is  applying  the 
usual  Benders'  scheme  to  solve  Problem  P2,  then  the  procedure  will  terminate  at  the  particular 
iteration  at  which  the  current  master  problem  yields  y*  (of  value  z*)  as  an  optimal  solution. 
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ABSTRACT 

Consider  a  regulated  monopolist  whose  current  profits  would  be  maximized 
if  they  could  charge  a  price  p,  where  p  exceeds  the  current  market  price.  By 
reducing  production  below  current  consumer  demand  the  monopolist  can  create 
an  illusion  of  a  shortage  and  induce  the  regulator  to  allow  a  price  increase. 
Conditions  are  given  for  which  the  production  rate  that  maximizes  the 
monopolist's  expected  discounted  profits  over  an  infinite  horizon  will  have  the 
property  that  the  amount  of  unsatisfied  consumer  demand  will  be  a  nonincreas¬ 
ing  function  of  vurrent  market  price. 


1.  INTRODUCTION 

Consider  a  monopolist  lacing  a  demand  curve  D(p).  If  the  cost  of  producing  x  units  is  cx 
then  the  firm  would  like  to  charge  the  profit  maximizing  price  p,  maximizing  the  revenue 
(p  -  r)  Dip).  Due  to  regulation  the  monopolist  cannot  immediately  attain  a  price  of  p.  How¬ 
ever,  by  producing  less  than  Dip),  the  demand  associated  with  the  price  p,  the  monopolist  can 
create  the  illusion  of  a  shortage;  this  shortage  induces  the  regulatory  agency  to  increase  the 
price.  Although  the  economic  grounds  for  the  regulator  to  base  his  pricing  decisions  on  the 
observed  level  of  excess  market  demand  may  be  scant,  regulatory  agencies  in  several  industries 
behave  in  this  fashion.  For  example,  the  degree  of  market  disequilibrium  in  the  natural  gas 
industry  is  a  crucial  factor  in  the  determination  of  the  regulated  price.  Owen  and  Brautigam 
note  [9],  that  "Producers  are  sometimes  alleged  to  ‘hold  back'  or  delay  production  until  regula¬ 
tors  raise  prices  in  the  future." 

Assuming  the  current  price  is  p  with  p  <  p  and  p  >  c,  the  monopolist  must  trade  off  the 
profits  lost  by  not  satisfying  a  portion  of  current  demand  against  the  future  profits  that  can  be 
gained  by  holding  back  production  in  order  to  induce  a  price  increase.  Our  approach  utilizes 
the  concept  of  continuous  time  and  semiMarkov  decision  processes.  (Markov  decision  models 
in  which  the  firm  controls  price  and  not  production  have  been  analyzed  elsewhere  (see  [3],  (4], 
and  (8].)  At  each  point  in  time  the  state  of  the  process  is  the  (regulated)  price  at  which  the 
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monopolist's  production  can  be  sold;  the  monopolist's  action  is  the  rate  at  which  the  product  is 
produced  or,  equivalently,  the  amount  of  consumer  demand  the  firm  chooses  to  leave 
unsatisfied.  The  regulator's  willingness  to  increase  price  depends  on  the  current  price  as  well  as 
the  consumer  demand  that  the  firm  leaves  unsatisfied.  Thus,  it  is  natural  to  attempt  to  deter¬ 
mine  whether  the  optimal  production  strategy  of  the  regulated  monopolist  increases  in  the  regu¬ 
lated  price  p  and,  more  imporuntly,  whether  the  excess  or  unsatisfied  consumer  demand  associ¬ 
ated  with  its  production  strategy  decreases  as  the  price  increases.  Our  goal  is  to  derive  condi¬ 
tions  which  ensure  that  the  amount  of  unsatisfied  consumer  demand  is  a  nonincreasing  function 
of  the  price. 

In  Section  2  we  describe  the  two  models  analyzed  in  this  paper.  Sections  3  and  4  are 
devoted  to  a  characterization  of  the  manner  in  which  the  firm's  optimal  production  strategy 
depends  on  parameters  such  as  the  current  price,  the  discount  factor,  and  the  length  of  the 
planning  horizon.  Finally,  Section  5  suggests  an  extension  of  our  model. 

2.  MODEL  DESCRIPTION 

In  both  models  to  be  considered  the  monopolist  observes  the  price  p  at  which  its  product 
can  be  sold.  This  price  is  set  by  the  regulatory  agency.  When  the  price  is  p,  demand  for  the 
product  occurs  at  an  instantaneous  rate  D(p).  While  the  price  is  p  and  the  monopolist's  output 
rate  is  y  <  f)(p),  the  monopolist  earns  profit  at  the  rate  (p  -  cly,  where  c  is  the  true  constant 
marginal  cost  of  production. 

Our  two  models  differ  in  the  manner  in  which  the  monopolist's  production  decision 
affects  the  evolution  of  the  product's  price.  In  model  1  we  assume  that  at  any  insunt  during 
which  the  price  is  p  and  the  amount  of  unsatisfied  consumer  demand  is  x,  the  price  increases  to 
p  +  1  in  a  time  that  is  governed  by  a  Poisson  process  with  rate  kip,  x).  Naturally,  that  amount 
X  of  unsatisfied  consumer  demand  satisfies  x  —  Dip)  -  y.  In  model  2  the  price  increases  from 
p  to  p  -f  fc,  it  -  1.  2.  3  . . .  .  with  probability  /(*)  after  a  time  that  is  governed  by  a  Poisson 
process  with  rates  X  (x).  For  both  models  we  assume  profits  arc  continuously  discounted  at  a 
rale  a  and  the  monopolist  wishes  to  maximize  the  expected  discounted  profit  earned  over  an 
infinite  horizon. 

The  firm  is  not  allowed  to  select  the  price  but  rather  is  restrained  by  the  regulatory 
agency.  Determining  the  firm's  marginal  cost  curve  is  no  easy  task  for  the  regulatory  agency 
and  in  this  case  the  agency  will  make  its  inferences  based  upon  the  firm's  response  to  the 
agency's  actions.* 

The  agency  presumes  that  the  firm  has  an  increasing  marginal  cost  curve  as  depicted  in 
Figure  1.  With  the  price  given  and  p  <  /’a  the  firm  will  set  its  production  so  as  to  equate  mar¬ 
ginal  cost  and  price.  Moreover,  it  is  clear  from  Figure  1  that  if  the  firm’s  actual  marginal  cost 
curve  were  as  shown,  then  excess  demand  iDip)  —  y)  decreases  with  price  while  production 
increases.  From  the  agency’s  perspective,  the  desired  price  is  Pn,  for  increasing  the  price 
beyond  Pr  will  not  cause  production  to  increase  and  excess  demand  vanishes  at  pr.  Thus,  the 
agency  will  allow  price  to  increase  until  excess  demand  vanishes  and  market  is  cleared. 


•See  D  P  Baron,  111.  for  another  approach. 
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In  fact,  the  firm  has  constant  marginal  cost  (and  <  Pk  for  otherwise  the  market  would  not 
clear  and  the  agency  would  revise  its  estimate  of  Pn  upward).  Consequently,  the  agency  eventu¬ 
ally  allows  the  price  to  reach  the  monopolistic  price. 

Finally,  the  speed  of  the  price  adjustment  varies  directly  with  the  disequilibrium  as  meas¬ 
ured  by  the  excess  demand  x  (see  assumption  2  below). 

3.  ANALYSIS  OF  MODEL  1 

We  begin  by  anlyzing  the  dependence  of  the  optimal  production  strategy  on  the  current 
price.  Defining  /  -  (0,  1,  2,  ...  )  and  ir  (p)  -  (p  -  c)D{p),  our  results  require  the  following 
assumptions; 

1.  X  (p.  x)  is  a  differentiable  function  of  x. 

2.  X(p.  X2)  >  X(p.  xi).  X2  >  xj. 

3.  X  (p,  0)  -  0.  p  €  /. 

4.  X(p  -I-  1,  x)  <  X(p,  x).  p  €  /. 

5.  X  (p  -I-  1.  X2)  -  X  (p  -I-  1,  X|)  <  X  (p,  X2)  -  X  (p,  Xi).  p  €  /.  X2  >  Xj. 

6.  X(p  -I-  1.  X2)/X(p  +  1.  X|)  <  X(p.  X2)/X(p.  Xi),  P  €  /,  X2  >  Xi. 
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7.  D{p  +  1)  <  Dip),  p  €  /. 

8.  There  is&p^l  such  that  trip)  ^  ir  ip)  for  all  p  €  /. 

9.  ir  (p  +  1)  -  TT  (p)  <  IT  (p)  -  IT  (p  —  1),  p  —  c  f  2 . p. 

10.  IT  (p  4-  1)  <  TT  ip),  p  p.  p  +  1 . 


Assumption  1  is  made  for  the  sake  of  convenience.  Assumption  2  reveals  that  the  larger 
the  excess  demand  the  greater  the  tendency  for  the  price  to  rise.  If  the  likelihood  that  the 
regulator  will  grant  a  price  increase  increases  with  the  observed  level  of  excess  demand  then 
Assumption  3  will  be  true.  This  is  true  in  the  natural  gas  industry  (see  [91).  Assumption  3 
states  that  the  firm  may  maintain  the  current  price  if  it  so  desires  by  producing  exactly  what  the 
consumers  demand.  Assumption  4  implies  that  at  higher  price  levels  it  is  more  difficult  for  the 
firm  to  induce  price  increases,  whereas  assumptions  5  and  6  imply  that  at  a  higher  price  an 
increase  in  unsatisfied  demand  has  a  smaller  effect  on  the  rate  at  which  the  price  increases.  We 
note  that  Assumptions  5  and  6  are  trivially  satisfied  if  A  (p,  x)  is  independent  of  p.  Assumption 
7  merely  asserts  that  the  demand  curve  for  the  monopolist's  product  is  downward  sloping. 
Assumptions  8-10  are  valid  for  a  variety  of  reasonable  demand  curves  including  Dip)  ~ 
ia.  b  >  0),  Dip)  =  p~*(k  >  1),  and  Dip)  ^  a  -  bp  ia.  b  >  0).  Finally,  we  make  the  rea¬ 
sonable  assumption  that  whenever  the  current  price  is  p,  the  monopolist  will  never  produce  at  a 
rate  exeeding  Dip).  This  is  reasonable  because  any  production  in  excess  of  Dip)  will  earn  no 
revenues  and  (by  Assumptions  2  and  3)  will  have  no  effect  on  the  future  price.  This  assump¬ 
tion  (coupled  with  Assumption  7)  allows  us  to  assume  that  the  set  of  possible  actions  for  each 
price  is  a  finite  set. 

We  now  define  F(p)  to  be  the  maximum  expected  discounted  profit  earned  over  an 
infinite  horizon  when  the  current  market  price  is  p.  It  follows  that 


where 


F(p)—  max  Jip.  x). 

Jip,  x)  -  {(p  -  c)  iDip)  -  x)  4-  X  ip,  x)  yip  +  l))/(\  ip,  x)  4-  a). 


Let  x(p)  be  the  largest  value  of  p  attaining  the  maximum  in  (1).  Our  goal  is  to  derive  condi¬ 
tions  which  ensure  that  xip)  is  a  nonincreasing  function  of  p;  that  is,  the  higher  the  price  the 
smaller  the  amount  of  consumer  demand  that  will  be  left  unsatisfied  by  the  firm. 

Our  analysis  of  the  behavior  of  x(p)  will  be  facilitated  by  the  study  of  a  modified  version 
of  the  infinite  horizon  problem  described  above.  Towards  this  end  we  let  V„ip)  be  the  max¬ 
imum  expected  discounted  profit  earned  by  the  firm  over  an  infinite  horizon  when  the  current 
price  is  p  and  at  most  n  price  increases  are  permitted.  It  follows  that  iV^ip)  equals  tr  (p)/a  if 
p  >  c  and  0  if  p  <  c) 

(2)  l'»(p)”  max  y,(p,  x)  n  >  1, 


where 


y,(p,  x)  -  {(p  -  c)  iDip)  -  x)  4-  X  ip,  x)  V„-x  ip  +  1)}/(X  (p.  x)  4-  «). 


NAVAL  RESEARCH  LOGISTICS  QUARTERLY 


VOL.  28.  NO.  3.  SEPTEMBER  1981 


‘4 


OPTIMAL  CONTROL  OF  PRICE  THROIJCH  RESTRICTED  PRODUCTION  467 

Our  main  result  will  require  the  following  lemmas.  The  trivial  proof  of  Lemma  1  will  be  omit¬ 
ted. 

LEMMA  I:  If  Assumptions  8  and  9  are  valid,  then  for  n  ^  0  and  c  < 

P  ^  p,  Ki(p)  >  ir(p  -  D/a. 

LEMMA  2:  If  Assumption  10  holds,  then  for  p  >  pand  n  >  0  f'.Cp)  ■■  J^ip.  0). 

PROOF:  The  result  is  clearly  true  for  n  —  0.  We  therefore  assume  the  result  to  be  valid 
for  n  -  1  and  verify  that  it  remains  valid  for  it.  To  prove  this  observe  that 

(a  +k(p,  x))7,(p.  0)*  (a  +X(p,  x))n{p)/a  (by  (2)1 

^  (p  -  c)  (Dip)  -  x)  +  X  ip.  x)ir  ip  +  l)/a  (by  Assumption  101 

-  (o  +  X  ip,  x))y,(p.  x)  (by  the  induction  hypothesisl. 

Thus,  the  induction  hypothesis  implies  that  J^ip,  0)  »  iripMa  so  the  above  inequalities  imply 
that  J„ip,  0)  >  J,(p.  x)  which  is  the  desired  result. 

LEMMA  3:  If  Assumptions  4,  7,  8,  9,  and  10  are  valid,  then  for  n  >  0  and 
c  ^  P  <  P  -  1. 

(3)  a  F„(p  +  1)  -  w(p)  <  a  F,(p)  -  ir(p  -  1). 

PROOF:  We  prove  (3)  by  induction.  For  n  —  0  (3)  reduces  to  Assumption  9.  We 

therefore  assume  that  (3)  is  valid  for  n  -  1  and  verify  that  it  remains  valid  for  n.  Define 

xin,  p)  to  be  the  largest  value  of  x  satisfying  V„ip)  -  y^(p,  x(n,  p)).  Let  x  -  x(n,  p  +  1). 

Since  0(p  +  1)  <  D(p),  x  is  a  feasible  action  when  the  current  price  is  p.  This  implies  that 

(4)  a  V„(p)  >  al(p  -  c)  (Dip)  -  x)  -F  X  (p,  X)  F,_j(p  +  1))/(X  (p,  x)  -F  a). 

(5)  a  V„ip  -F  1)  ••  o{(p  +  1  -  c)  (Dip  -F  1)  -  Jc) 

+  X(p  +  1,  x)  y„-\(p  +  2))/(X(p  +  1,  X  -F  a)  (by  (2)) 

<  a((p  -F  1  -  c)  (Dip  -F  1)  -  x)  -FX  (p,  x)  F,_i(p  -F  2))/^  (p,  x)  +  a) 
[by  Lemma  1  and  Assumption  41. 

Together  (4)  and  (5)  imply 

(6)  «(l^n(P  +  1)  -  y^ip))  <  «l(P  +  1  -  c)  (Dip  -F  1)  -  x)  -  (p  -  c)  (Dip)  -  x) 

+  X(p,  x)[F,_,(p  +  2)  -  Vn-\ip  +  Dl)/(X(p,  x)  -Fo). 

Note  that 

(7)  (p  +  1  -  c)(0(p  +  1)  -  x)  -  (p  -  c)  (Dip)  -  x)  -  ir(p  -F  1)  -  '>r(p)  -  X 

<  ir  (p)  -  w  (p  -  1)  (by  (9)  and  p  -F  1  <  pi. 

Also  observe  that  for  c  ^  p  <  p  —  1  the  induction  hypothesis  implies  that  o  ( i  (p  -F 

2)  -  Kf-i(p  +  D)  <  IT  (p  +  1)  -  ir  (p)  while  Lemma  2  and  Assumption  8  imply 

a(F,_,(p  -F  1)  -  F,_,(p))  -  ir(p  -F  1)  -  ir(p)  <  ir(p)  -  ir(p  -  1). 
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Thus,  for  c  <  <  p  —  1  it  follows  that 

(8)  a(K,_,(p  +  2)-  +  D)  <ir(p+  l)-7r(p)  <ir(p)-ir(p-  1). 

Together  (6)-(8)  imply  that 

a(K,(p  +  1)  -  <  ir(p)  -  ir(p  -  1)  c  <  p  <  p  -  1 

which  is  the  desired  result. 

Lemma  2  implies  that  for  p  >  p  the  firm's  optimal  policy  is  to  satisfy  all  consumer 
demand  by  producing  at  a  rate  Dip).  We  now  show  that  for  n  >  0  and  c  <  p  <  p  —  2, 
x(n,  p)  >  x(n,  p  +  1).  Coupling  this  result  with  Lemma  2  we  see  that  for  all  n  and  p  c 
the  firm's  optimal  level  of  unsatisfied  demand  (the  amount  of  market  disequilibrium)  is  a 
nonincreasing  function  of  p. 

THEOREM  1;  If  (I)-(IO)  are  valid,  then  for  n  >  0  and  c  <  p  <  p  -  2. 

(9)  X  (n.  p  +  1)  <  xin,  p). 

PROOF;  Observe  that  (9)  will  follow  if  we  can  prove  that 

(10)  j„{p  +  1.  xj)  -  j„(p  +  1.  X,)  ^  0^  y„(p.  X2)  -  y,(p.  Xi)  >  0 
holds  for  /I  ^  0,  p  +  1  <  p  —  1,  and  Dip  +  1)  >  X2  ^  x,. 

We  now  proceed  to  verify  (10).  After  some  algebraic  manipulations  J„ip  +  1.  *2)- 
y,(p  +  1.  xi)>  0  can  be  shown  to  be  equivalent  to 

(11)  a  F,_,(p  +  2)  -  ir(p  +  1)  >  (p  +  1  -  c)!a(x2  -  Xi) 

+  X2\  (p  +  1,  X|)  -  X|X  ip  +  1,  X2))/(\  (p  +  1,  X2)  -  X  (p  +  1,  Xj)). 


By  (11),  (10)  will  hold  if  the  left  side  of  (11)  is  a  nonincreasing  function  of  p  and  the 
right  side  of  (11)  is  a  nondecreasing  function  of  p.  Since  p  +  2  <  p.  Lemma  3  implies  that  the 
left  side  of  (11)  is  nonincreasing  in  p.  Since  Assumptions  2  and  S  imply  that 
a  (p  +  1  -  c)  (x2  -  xi)A  (p  +  1.  X2)  -  A  (p  4-  1.  Xj))  is  nondecreasing  in  p,  the  right  side  of 
(11)  will  be  nondecreasing  in  p  if 

X2X  (P  -f  1.  Xi)  -  X|X  ip  +  1,  X2)  ^  X2X  ip,  X|)  -  X|X  (P2,  X2) 

X  (p  +  1.  X2)  -  X  (p  +  1.  X|)  X  ip,  X2)  -  X  ip,  X]) 

Upon  simplification  (12)  is  seen  to  be  equivalent  to  (x2  -  X|)  {X  (p  +  1,  xi)X(p,  X2)  - 
X(p  +  1,  x2)X(p,  xj))  ^  0.  Since  X2  >  X|,  the  last  inequality  follows  from  Assumption  6. 
This  completes  the  verification  of  (11),  and  the  proof  of  Theorem  1  is  complete. 

The  following  Lemma  relates  V„ip)  to  K(p). 

LEMMA  4;  For  all  p  lim,_  F,(p)-  K(p). 

PROOF;  Since  nip)  is  bounded  above  by  ir(p),  the  result  follows  directly  from  Lemma 
1  of  (2J. 
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Lemmas  2  and  4  imply  that  for  n  ^  A  V{p).  Theorem  1  plus  Lemmas  2  and  4 

now  imply  that  V(p)  -  jip.  0)  ip  ^  p),  and  x(p  +  1)  <  x(p),  (c  ^  p  ^  p  -  2). 

Let  x^ip)  and  Vaip)  denote  the  dependence  of  these  quantities  on  the  discount  factor®. 
We  now  show  that  is  a  nonincreasing  function  of  the  discount  factor  a.  In  order  to 

prove  this  result,  we  first  consider  a  finite  horizon  semiMarkov  decision  process  formulation  of 
the  problem.  We  use  the  technique  of  Lippman  [6]  in  which  the  system  is  observed  and  the 
production  rate  can  be  changed  only  at  discrete  points  in  time,  namely  those  corresponding  to 
price  increases  and  certain  null  events.  These  null  events,  which  do  not  change  the  current 
price,  are  introduced  in  order  to  make  the  time  between  observations  independent  of  the  state 
and  action  and  they  render  the  problem  equivalent  to  a  discrete  time  Markov  decision  process. 
After  analyzing  the  discrete  time  Markov  process  we  will  let  the  horizon  length  go  to  infinity 
and  exploit  the  equivalence  of  this  formulation  to  the  original  continuous  time  problem  to  get 
results  for  our  original  problem,  that  is,  the  inihnite  horizon  problem  in  which  the  production 
rate  can  be  changed  at  any  point  in  time. 

We  now  consider  the  system  to  be  observed  only  at  times  when  price  increases  or  null 
events  are  observed:  The  time  between  observations  is  exponentially  distributed  with  parame¬ 
ter  A  -  sup  kip.  x)  —  X  (0.  0(0)),  where  the  last  equality  follows  from  Assumptions  2  and 

p€/ 

xiD(p) 

4.  The  state  of  the  system  is  p,  the  current  price,  and  the  action  is  x  if  production  is  occurring 
at  a  rate  Dip)  ~  x.  The  probability  that  the  next  observation  point  is  occasioned  by  a  price 
increase  is  X  ip.  x)/A  while  the  probability  that  the  next  observation  point  is  a  null  event  is 
1  -  X  (p.  x)/A. 

Let  be  the  maximum  expected  profit  earned  by  the  firm  when  n  observation 

points  remain,  the  current  price  is  /,  and  the  discount  factor  is  a .  Then 

itVoJp)~0) 

(13)  max  G„„ip,  x)/(A  +a),  «  >  1, 

0<x$O(pt 

where 

(r„,o(p.  x)  -  p(0(p)  -  x)  +  X  ip.  x)A  M',,,  „(p)  -t-  A  If",.,  „(p) 
and 

ikw„„ip)^  iy„.„ip  +  \)-  w„.jp). 

Let  x„a^P)  be  the  largest  value  of  x  attaining  the  maximum  in  (12).  We  now  prove  that 
if  Assumptions  1,  2,  3,  7,  8,  and  10,  and  nip  +  1)  ^  ir(p)(p—  c.  c  1.  ...  p),  are  valid 
then  x„a(p)  is  a  nonincreasing  function  of  «.  By  Theorem  1  of  I5l  and  Theorem  1.1  of  (lOl 
this  wili  imply  that  x^^ip)  ^  x^^ip)  ia2^  a^)  which  Is  the  desired  result.  Before  proving  this 
result  we  require  the  following  lemma. 

LEMMA  5;  If  Assumption  10  holds,  then  for  «  ^  0  and  p  >  p,  W„aip)  “  0). 

PROOF:  The  proof  is  virtually  identical  to  the  proof  of  Lemma  2  and  is  therefore  omit¬ 
ted. 
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LEMMA  6:  If  Assumptions  I,  2,  3,  7,  8,  and  10  are  valid  and  ir  (/>  +  1)  >  ir  (p)  is  valid 
for  f  <  /»  <  p,  then  A  W^„{p)  >0(c<p<p-l). 

PROOF:  We  prove  the  result  by  induction  on  n.  For  n  *  1  the  result  follows  immedi¬ 
ately  from  IT  (p  -I-  1)  >  TT  (p).  Assuming  the  result  for  «  -  1  and  letting  x  *  Jf„a(p  +  D.  (13) 
and  X  -  D(p  -P  1)  -I-  Dip)  <  Dip)  imply 

(14)  (A  -l-a)AlF,„(p)  >  C„.„(p  -I-  1,  x)  -  G„„(p,  x  -  Dip  +  \) -^  Dip)) 

-  (A-\(p,  X-  Dip+\)  +  Z)(p))A)F,_,,.(p)  -hXip+  1.  x)A»f',_,„(p  -h  1). 

For  c  <  p  <  p  -  1  the  nonnegativity  of  the  last  expression  follows  from  the  induction 
hypothesis.  To  complete  the  proof  it  therefore  suffices  to  show  that  (p)  >  f*'n.a(p  -  D. 
To  prove  this,  note  that  ir  ip)  >  ir  (p  -  1)  the  induction  hypothesis,  and  Lemma  5  imply 

-  1)  -  (p  -  c  -  1)  iDip  -  1)  -  x,(p  -  D) 

-t-X(p-  1.  x„aip-  l))lF,_,„(p) 

+  (A-X(p-  1,  x^^ip-  D)  l*',-i,„(p-  1))/(A  +a) 

<  {7r(p)  -I-  A  <,(p))/(A  +a)  -  )V„  aip) 


This  completes  the  proof  of  Lemma  6. 

We  can  now  prove 

THEOREM  2:  If  Assumptions  1,  2,  3,  7,  8,  and  10  are  valid  and  nip  +  1)  ^  nip)  is 
valid  for  c  <  p  <  p  -  I,  then  fora2  >  aj  and  c  <  p  <  p  -  1  x„„^ip)  <  x^^^ip). 

PROOF:  By  reasoning  analogous  to  that  used  to  justify  (10)  it  suffices  to  prove  that 
Gna^ip.  Xi)  -  C„.„j(p,  X,)  <  G„„^ip.  Xi)  -  G,,a,(p.  X,) 
holds  foraj  >  «i,  and  Xj  >  Xj,  and  c  <  p  ^  p  —  1.  By  (13)  this  inequality  is  equivalent  to 

(15)  (A  ip.  xj)  -  A  ip.  X,))  (A  „,(p)  -  A  .„,(p))  <  0. 

By  Assumption  2,  inequality  (IS)  will  hold  if 

(16)  A  V„_^  „^ip)  <  A  F,_,  „j(p),  tti  >  a,,  c  ^  p  ^  p  -  1. 

To  prove  (16)  by  induction  note  that  for  » -  1  (16)  follows  from 

1/(A  -I- a 2)  <  1/(A  -Ka,).  Assuming  that  (16)  is  valid  for  n  -  1  we  can  verify  it  for  n  by 

observing  that 

(A  +a)  )P„,„,(p)  >  G,.„,(p  -t-  1,  X,)-  0,0,,  (p,  xi) 
and 

(A  -I- a)  W^ajip)  <  G„„j(p  +  I,  X,)  -  G,.„j(p  +  1.  X2), 
where  xj  -  x,.„(p  +  1)  and  X2  -  x,„(p). 
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These  inequalities  imply  that 

(A  +«)|Al^',,„,(p)- Alf'.,„^(p)l  >  |X(p  +  1.  x,)(Alf',_,  „,(p+  1) 

+  D) 

+  (A  -  A  ip.  xi)  (A  -  A  H',-,  „>))l 


where  the  last  inequality  follows  from  the  induction  hypothesis.  This  completes  the  proof  of 
Theorem  2.  By  the  remarks  following  Lemma  4  the  hypotheses  of  Theorem  2  also  imply 

We  now  characterize  the  dependence  of  x„„ip)  on  n.  More  specifically  we  show  that  the 

hypotheses  of  Theorem  2  imply  that  x„+i  „(p)  ^  x„  „ip)  is  valid  for  c  <  p  <  p  -  1.  If  we 

assume  that  the  firm  wishes  to  maximize  its  expected  discounted  profits  over  a  finite  horizon  of 
length  T  <  oo  and  we  define  x,a(p)  to  be  an  optimal  action  when  a  time  /  remains.  Theorem  4 
of  (61  and  the  above  result  will  enable  us  to  conclude  that  „(p)>  x,^  „ip)  for  >  t\. 

THEOREM  3:  If  the  hypotheses  of  Theorem  2  are  valid,  then  for  /r  >  1  and 

c  <  P  <  p  -  1  x,+  i  ,  (p)  ^  x,„{p). 

PROOF;  The  result  will  follow  if  we  can  prove  that 

G/i  +  l.o^P-  Xi)  -  G,  +  ,  „(p.  X,)  >  C,.a(p.  X2)  -  C„,„(p.  X,) 

holds  for  n  >  I,  xj  >  Xi,  and  c<p<p-l.  By  (13)  this  inequality  is  equivalent  to 
(A(p,  xjl-Alp.  x,))(A  lF„o(p)  -  Alf'„_|  „(p))  >  0.  By  Assumption  2,  the  last  inequality 
will  follow  if 

(17)  Alf'„.„(p)  ^  AlF^.iJp).  n^l.  c<p<p-l. 

The  proof  of  (17)  is  similar  to  the  proof  of  (16),  and  is  therefore  omitted. 

3.  ANALYSIS  OF  MODEL  2 

We  now  characterize  the  dependence  of  the  optimal  production  strategy  for  Model  2  on 
the  current  price.  Our  results  require  the  following  assumptions'. 

11.  A  (X2)  ^  A  (x]).  Xp  >  X| 

12.  A  (x)  is  differentiable 

13.  A(0)-0 

14.  x-JiKixi)  +a)  ^  Xi/(A(xi)  +a),  xj>  X| 

15.  There  is  a  p  €  /  such  that  ir  (p)  ^  ir  (p)  for  all  p  €  /. 

16.  For  p  6  /.  7r(p)  is  a  concave  function  of  p,  that  is  nip  +  1)  -  nip)  <  jr(p)  - 
nip  -  I). 
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Assumption  14  is  the  only  condition  which  requires  any  explanation.  If  X  (jr)  is  concave,  then 
Assumption  14  will  be  valid.  To  show  this  define  h(x)  —  x/(X(x)  +<*).  Then  for  any  x©  >  0 
h'(xo)“  X(xo)  +  o  -  xqX’  (xo)/(X(xo)  +a)^.  Together  Assumption  13  and  the  concavity  of 
X(x)  imply  that  X(xo)  >  X(0)  +  xqXTxo)-  X(^'(xo).  This  shows  that  h'(xo)  >  0,  which 
implies  Assumption  14. 

We  now  define  Tip)  and  T„(p)  to  be  the  analogs  (for  Model  2)  of  Vip)  and  V^ip). 

Then 

(18)  Tip)  -  max  Hip.  x). 

0<x<D(o) 

where 

Hip,  x)  —  {(p  —  c)  iDip)  —  x)  +  X  (x)  ^  fik)  Tip  +  k)/ (X  (x)  +  a) 

*-i 

and 

(19)  T„ip)~  max  H^ip,  x). 

0<x</)(/i) 

where 

H^p,  x)-  {(p-  c)(/)(p)~  x)  +X(x)  *£  /(ik)r,_,(p  +  X))/(X(x)  +a). 

*-i 

We  also  define  x(p)  and  x„ip)  to  oe  the  largest  values  of  x  attaining  the  maximum  in  (18)  and 

(19) ,  respectively.  The  proof  of  the  following  lemma  is  virtually  identical  to  the  proof  of 
Lemma  2  and  is  therefore  omitted. 

LEMMA  7:  If  Assumptions  15  and  16  are  valid,  then  for  n  >  0  and  p  >  p 

r,(p)-  HM  0). 

As  in  Section  3,  we  now  focus  our  attention  on  showing  that  for  c  ^  p  <  p  —  1, 
x„(p  +  1)  <  x,(p).  Before  proving  this  result  we  require  the  following  Lemma: 

LEMMA  8;  If  Assumptions  15  and  16  are  valid,  then  for  p  >  c  and  n  >  0. 

(20)  a  ^  /(*)r,_,(p  +  1  +  X)  -  ir(p  +  1)  <  a  £  /(X)r,-,(p  +  fc)  -  ir(p). 

*-i  *-i 

PROOF:  It  sufficies  to  show  that  for  each  k  >  1 

(21)  «T,-,(p  +  1  +  X)  -  ir(p  +  1)  <  ar,_,(p  +  X)  -vip). 

For  n  -  1  (21)  reduces  to  Assumption  16.  We  therefore  assume  that  (21)  holds  for 
n  -  1  and  verify  that  (21)  holds  for  n.  Letting  x  -  x,(p  +  1  +  X)  we  see  that  a  T„ip  + 
1  X)  -  //,(p  +  1  +  X,x)  and 

aT^ip  +  k)  >  o  //,(p  +  X,  x). 
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The  last  two  statements  imply 

a[T„(p  +  I  +  fc)  -  T„(p  +  /k)l  <  a|(p  +  k  +  1-  c)  (D(p  +  it  +  I)  -  x) 

(p  +  k  -  c)  (Dip  ■¥  k)  -  x) 

+  X(x)  X  +  k  +  J  +  1)  -  T^-\ip  +  k  +yll/(X(x)  +o). 

i-\ 

(p  +  Kl  +  I  -  c)  (Dip  +  A:  +  1)  -  x)  -  ip  +  k  -  c)  (Dip  +  k)  -  x) 

—  ir(p  +  A:  +  1)  —  IT  (At)  —  x  <  ir(p  +  I)  —  w(p)  (by  Assumption  16) 


a(T„-iip  +  k  +  J  +  1)  ~  T„-iip  +  k  +  J))  ^  ir  ip  +  \)  -  IT  ip), 

[by  the  induction  hypothesis) 

(22)  implies  that 

aiT„-iip  +  1  +  Ac)  -  r,_|(p  +  A))  ^  Trip  +  1)  -  ir  ip), 
which  is  equivalent  to  (21). 

THEOREM  4:  For  «  ^  0  and  c  <  p  <  p  —  2 

(23)  x,(p  +  1)  <  x„ip). 

PROOF:  The  proof  is  similar  to  the  proof  of  Theorem  1.  As  before,  it  suffices  to  demon¬ 
strate  that  for  Dip  -F  1)  >  X2  5#  X|,  p  <  p  —  2,  and  n  >  0 

H„ip  +  1.  xj)  -  H„ip  +1,  X,)  >  O-^  H„ip,  xj)  -  //,(p,  X,)  >  0. 

To  show  this,  note  that  H„ip  +  1,  x^)  —  H„ip  +  \,  Xj)  >  0  may  be  shown  to  be  equivalent  to 


(24) 


«  X  /(A)  r,_,(p -F  A  -F  1)  -  w(p -F  1) 


(X  (X2)  -  X  (xj)) 

^  (p  +  1  -  c)  (x2(X  (x,)  -Fa)  -  X|(X  (X2)  -Fa)). 


By  Lemma  8  the  left  side  of  (24)  is  a  nonincreasing  function  of  p.  Since  Assumption  14 
implies  that  the  right  side  of  (24)  is  a  nondecreasing  function  of  p  it  follows  that  (24)  implies 
^ii(p-  •*'2)  “  H„ip,  X|)  >  0.  This  completes  the  proof  of  the  theorem. 

As  in  Section  2,  it  easily  follows  that  limlFnlp)-  Wip)  and  that  for  n  ^  p, 

W„(p)—  Wip).  Together  with  Theorem  4  plus  Lemma  8,  these  facts  imply  that 
x(p)  >  x(p  -F  1)  (c  <  p  <  p  -  2)  and  that  in  the  unmodified  infinite  horizon  version  of 
Model  2  the  firm  should  produce  at  a  rate  Dip)  for  p  ^  p. 

Unfortunately,  all  efforts  to  determine  how  the  optimal  production  policy  for  Model  2 
depends  on  the  discount  rate  and  the  length  of  the  horizon  have  been  unsuccessful. 
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5.  EXTENSIONS  AND  SUGGESTIONS  FOR  FUTURE  RESEARCH 

One  obvious  defect  of  our  model  is  that  decreases  in  price  are  not  allowed.  A  model 
which  allowed  for  the  price  to  decrease  from  p  to  p  -  1  in  a  time  governed  by  a  Poisson  pro¬ 
cess  with  ratep  (p.  x)  (withp  (p.  X2)  <  p  (p.  X|))  as  well  as  increase  from  p  to  p  -l-  1  in  a  time 
governed  by  a  Poisson  process  with  rate  X  (p.  x)  (X  (p.  x^)  >  X  (p,  xi))  would  be  of  obvious 
interest.  Unfortunately,  we  have  been  unable  to  obtain  characterizations  of  the  optimal  produc¬ 
tion  policy  for  such  a  model. 
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ABSTRACT 

Momonl  and  maximum  likelihood  esiimates  (m  1  c  'si  arc  mvcsiigaicd  lor 
nonparameirie  and  paramcirie  models  lor  a  single  server  queue  observed  over  a 
random  lime  hon/on.  namely,  up  lo  ihc  »ih  departure  epoch  Also,  m  I  c 's  of 
ihe  mean  inierarrival  lime  and  mean  service  lime  in  an  V/  MM  queue  ob¬ 
served  over  a  lived  lime-imerval  are  studied  l.imii  disiribuiions  of  these  esii- 
maics  are  ohiained  wiihoui  imposing  sieady  stale  assumpiioiis  on  ilie  queue- 
si/e  or  walling  lime  processes 


I.  INTRODUCTION 

The  theory  of  queues  as  developed  so  far  is  largely  a  descriptive  theory,  namely,  it  is  con¬ 
cerned  with  the  probabilistic  structure  of  the  models  and  the  behaviour  in  finite  time  r  as  well 
as  in  the  limit  as  f  —  «>  of  the  processes  arising  from  these  models.  Relatively  less  has  been 
done  on  a  prescriptive  theory  dealing  with  the  statistical  analysis,  design  and  control  of  queue¬ 
ing  systems.  Yet  from  the  practical  point  of  view  these  latte"  aspects  are  very  important.  Thus, 
for  example,  the  management  of  a  service  facility  subject  ti>  congestion  and  wishing  to  design 
an  efficient  queueing  system  should  be  in  a  position  to  estimate  the  various  parameters  of  the 
model  on  the  basis  of  data  collected  at  the  facility. 

The  earliest  paper  on  problems  of  statistical  inference  from  queueing  models  seems  to  be 
that  of  Clark  [4]  who  investigated  the  problem  of  maximum  likelihood  estimation  of  the  param¬ 
eters  of  an  M/Ml\  queue  in  equilibrium.  While  this  paper  is  of  historic  importance,  the  more 
substantial  investigation  of  these  problems  was  carried  out  by  Cox  (S]  and  Wolff  (14).  The 
work  of  Cox  contains  several  ideas,  but  no  major  results  seem  to  emerge.  Wolff  discussed 
maximum  likelihood  estimation  and  likelihood  ratio  tests  for  a  class  of  ergodic  queueing  models 
which  give  rise  to  birth  and  death  processes  (including  the  queue-length  process  in  Ml  Ml  sand 
related  systems).  Since  these  processes  are  continuous  time  Markov  processes,  Wolff  was  able 
to  derive  limit  distributions  of  the  estimates  and  test  statistics  as  a  direct  application  of 
Billingsley's  [3]  results  for  Markov  processes.  The  Markov  property  of  the  queue-length  pro¬ 
cess  and  the  assumption  of  steady  state  are  the  essential  features  in  WolfTs  study  as  they  are  in 
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Billingsley's.  For  further  work  on  the  A//A//1  queue  see  Harris  I8l,  Jenkins  (101,  and  Samaan 
and  Tracy  [13).  Benes  [21  treated  the  estimation  problem  for  the  A//A//00  system.  Goyal  and 
Harris  [7]  obtained  maximum  likelihood  estimates  of  the  parameters  of  a  queueing  system  with 
Poisson  arrivals  and  state-dependent  service,  again  assuming  steady  state.  Henningsen  [9] 
investigated  the  A//G/1  queue  in  equilibrium  and  obtained  estimates  for  the  arrival  rate  and 
the  parameters  of  the  service  time  density.  Crane  and  Lemoine  [6]  have  applied  simulation 
techniques  to  the  problem  of  estimating  the  steady  state  mean  waiting  time  in  a  single  server 
queue.  In  their  recent  monograph,  Basawa  and  Prakasa  Rao  [1]  discuss  examples  of  inference 
for  systems  such  as  MIEJ\  and  EJ Ml\^  again  within  the  framework  of  Billingsley  (3). 


The  objective  of  this  paper  is  to  consider  a  single  server  queueing  model  and  derive  'rea¬ 
sonable'  estimates  for  the  inierarrival  and  service  time  distribution  functions  (d.f.'s)  and  their 
means.  The  processes  arising  from  this  model  are  not  always  Markovian,  nor  do  we  assume  the 
existence  of  steady  state.  Our  estimates  are  not  necessarily  maximum  likelihood  estimates,  but 
have  a  simple  structure  of  intuitive  appeal  and  reduce  to  m.l.e.’s  in  special  cases.  The  estimates 
of  the  means  are  moment  estimates  and  those  of  the  d.f.'s  are  empirical  d.f.'s.  Limit  distribu¬ 
tions  of  the  estimates  are  obtained  using  direct  and  simple  arguments. 


In  Section  2  we  study  the  properties  of  moment  estimates  of  the  interarrival  and  service 
time  means,  and  also  the  estimates  of  the  corresponding  d.f.'s  in  a  G/G/1  queueing  system 
observed  over  a  random  time  horizon  (0,  D„l,  where  D„  is  the  nth  departure  epoch.  The  limit 
distributions  of  these  estimates  are  obtained  without  imposing  any  restrictions  on  the  traffic 
intensity.  Section  3  is  concerned  with  the  maximum  likelihood  estimation  of  the  parameters  of 
a  G/G/l  queue,  using  the  same  sampling  plan  as  in  Section  2.  In  Section  4  we  discuss  m.I.e.'s 
of  the  interarrival  and  service  time  means  in  an  Ml  MU  queue  observed  over  a  nonrandom 
time-interval  (0,  /),  and  study  their  properties  as  r  — *  ■»,  again  without  any  restrictions  on  the 
traffic  intensity.  The  problem  of  hypotheses  testing  will  be  considered  in  a  forthcoming  paper. 


NOTATION:  We  shall  use  to  denote  convergence  in  distribution.  The  normal  ran¬ 
dom  variable  with  mean  ft,  and  variance  <r^  will  be  denoted  by  W|<iu ,  <r^),  and  the  random  vec¬ 
tor  having  the  bivariate  normal  density  with  means^iiA^z  and  variance-covariance  matrix  £  will 


be  denoted  by 


"■11' 


As  usual,  a.s.  means  almost  surely. 


2.  THE  G/G/l  QUEUE- MOMENT  ESTIMATES 

Consider  a  single  server  queueing  system  in  which  the  interarrival  times  [u^,  k  >  1)  and 
the  service  times  {v^,  k  ^  1)  are  two  independent  sequences  of  independent  and  identically 
distributed  nonnegative  random  variables  with  d.f.'s  F  and  G  respectively.  Assume  that  the 
moments 

(1)  Eiu,,)  “  a,  Var{«n)  — cri^ 

(2)  £(v*)  -  6,  Var(vj)«<r2^ 

are  all  finite.  The  traffic  intensity  of  the  system  is  then  p  —  F/a,  also,  let  v  -  maxd,  p).  We 
assume  that  the  initial  customer  arrives  at  r  •  0.  Our  sampling  scheme  is  to  observe  the 
phenomenon  until  the  first  n  customers  have  departed  from  the  system  and  note  the  service 

times  of  these  n  customers,  say  (v,.  v; . v„).  Let  the  nth  departure  epoch  be  Z7„,  and  also 

observe  the  interarrival  times  of  all  customers  who  arrive  during  (0.  D„]\  thus,  we  obtain 
<Mi.  U2 . where 
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(3)  (D„)  -  max  {fe;  M|  +  Uj  +  . . .  +  «*  <  D„). 


Clearly,  ^  n.  For  the  means  a,  b  we  propose  the  estimates 


(4) 


I 


I 

n 


Iv,. 


it  should  be  observed  that  the  estimate  b„  is  the  usual  sample  mean,  whereas  a„  is  based  on  a 
random  number  of  observations.  It  would  be  interesting  to  compare  a„  with  the  ordinary  sam¬ 
ple  mean 


(5) 


1 

n 


It  turns  out  that  asymptotically  a„  and  a*  display  similar  behaviour,  namely,  they  are  both  con¬ 
sistent  estimates  of  a  and  asymptotically  normal,  with  a„  having  a  smaller  variance.  The  obvi¬ 
ous  independence  of  u*  and  b„  is  used  to  establish  the  asymptotic  independence  of  the  esti¬ 
mates  (4).  These  properties  are  proved  in  Theorem  I  below.  We  need  the  following  prelim¬ 
inary  result,  which  will  be  proved  in  the  appendix. 


LEMMA  1;  As  n  — *  «>, 

(6)  —  ‘  T)  in  probability. 


THEOREM  I:  As  w  —  «>, 
(i)  a„  —  a,  b„  — ♦  Aa.s.,  and 


yfn  (a„  -  a) 

0 

‘T.Vrj  0  1 

(ii) 

yfn  (b„  —  b) 

/V, 

0 

> 

0  o-j^j 

PROOF;  (i)  The  strong  law  of  large  numbers  gives 

(7)  i  V,  ^  A  a.s. 

n  \  /7  , 

Since  (6)  implies  that  N4(D„)  t  <»  as  w  — * «»,  we  also  obtain 


as  w  — '  oo.  We  have  thus  proved  both  statements  in  (i). 

(ii)  We  first  consider  the  limit  distribution  of  yfn  (a*  —  a),  -Jn  (b„  —  b),  where  a*  is  the 
sample  mean  (S).  On  account  of  the  independence  of  the  sequences  {u^)  and  (v;)  we  have 


Vn (a*  -  a) 

IN  ®  1 

yfn  (b„  -  b) 

by  the  central  limit  theorem.  The  difficulty  is  that  our  estimate  a„  is  based  on  observations 
where  (3)  shows  that  depends  on  D„  and  hence  on  |v*).  However,  we  shall  show  that  this 
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dependence  of  a„  and  b„  disappears  in  the  limit  as  //-♦«>.  The  desired  result  (ii)  will  be 
proved  via  the  Cramer- Wold  device,  if  we  show  that  for  arbitrary  real  numbers  o ,  /3, 


aVn  (a„  -  a)  +  fi-Jn  (b„  ~  b)  ^  Nf  0,  - -t-  . 


To  this  end,  we  write  the  left  side  of  (10)  as 


11/2  U/2 

{a*  -  a)  +  pyfit  (b„  -  b)  +  a  yfn  (a„  -  a)  -  —  (a*  -  a)  . 

The  limit  distribution  of  the  term  within  the  hrst  pair  of  brackets  in  (11)  is  given  by  (10)  on 
account  of  (9).  Therefore,  the  desired  result  (10)  will  follow  if  we  show  that  the  term  within 
the  second  pair  of  brackets  in  (11)  converges  to  zero  in  probability.  Now 


1/2 

1 

1/2 

V 

Ml 

.V 

A 

"  f.i 

V  /vr 


r 


First,  we  note  that  by  the  Central  Limit  theorem  applicable  to  random  sums  (see  Billingsley 
(31)  it  follows,  in  view  of  (6),  that  as  n  —  <», 


(13) 

and  that 

(14) 


X  («,  -  a)-^  A',(0.  a, 2) 

I 


I  (m;  -  a)  -  ^  t  («,  -  a)  -  0 


■sd 


in  probability.  Using  (13),  (14)  and  (6)  we  see  that  the  right  side  of  (12)  converges  to  zero  in 
probability  as  a  —  <».  Thus,  (10)  is  proved  and  the  proof  of  Theorem  1  is  complete. 

For  the  d.f.'s  Fand  G  the  natural  estimates  are  the  empirical  d.f.’s 

(15)  F„(u)~  I6(m  -  a,).  djv)~  -  fdlv-  v,) 

,  a  , 

where  6 (x)  -  0  for  x  <  0  and  —  1  for  x  >  0.  The  asymptotic  properties  of  these  estimates 
are  given  by  the  following  theorem.  Its  proof  is  similar  to  that  of  Theorem  1  and  is  therefore 
omitted. 

THEOREM  2:  For  all  u  >  0,  v  >  0,  we  have  as  a  —»  «» 

(i)  F„{u)  —  F(u).  C„(v)  —  G(v)  in  probability,  and 


yfn  (F„(u)  ~  F[u))  0 

y/Hid.M  -  C(v))  0  '  0 
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where 

(16) 


F(u)  [1  -  F(u)].  G(v)(l  -  G(v)l. 


3.  THE  G/G/1  QUEUE- MAXIMUM  LIKELIHOOD  ESTIMATES 

In  this  section  we  assume  that  the  d.f.'s  F  and  G  of  the  interarrival  times  and  service 
times  are  absolutely  continuous  with  continuous  densities  f{u\9)  and  ;(v;  0),  where  / and  g 
are  known  functions  of  unknown  real  parameters  0  and  0.  For  simplicity  of  presentation  we 
assume  that  0  and  0  are  scalars;  the  case  of  vector  parameters  can  be  treated  in  an  analogous 
manner.  Consider  the  problem  of  maximum  likelihood  estimation  of  0  and  0.  Under  the  sam¬ 
pling  scheme  of  Section  2  the  likelihood  function  is  given  by 


.(/.«)-  [i/(«o«)j  jn«(v,;0)  (1  -  0)1 


where  X„  =  X„(D„)  —  D„  -  ^  u,.  The  only  factor  that  causes  any  difficulty  (in  the  sense  of 

I 

yielding  simple  estimates)  in  (17)  is  1  -  F{X„\0)^  which  corresponds  to  the  incomplete  arrival 
interval  when  sampling  is  terminated  at  the  epoch  D„.  Consider  instead 


'(/.«)“  II /(«,;«)|  I  n  «(v/;  0)  . 


which  can  be  viewed  as  an  approximation  to  L,(/.  g)  in  a  sense  to  be  explained  later.  Let  0  ® 
and  0^  denote  the  likelihood  equation  estimators  (see  Rao  [121)  of  0  and  0  based  on  L°(/,  ;); 
thusO^,  0„  are  the  roots  of  the  equations 

y 

(•9)  Z l08 “  0.  z  >08  “  0. 

(  I  wp 

The  following  theorem  states  the  asymptotic  properties  of  S?,  0 

THEOREM  3:  Under  appropriate  regularity  conditions  of  /and  g  (see  Rao  [121,  for 


ol  0 


example) 

we  have  as  n  - 

(i) 

0:- 

(ii) 

yfn  (0^-  0) 

where 

(20) 

cr«  -  I4 

-  fjl-losy  •  *j  I 


PROOF:  The  stated  properties  of  0  °  follow  from  the  classical  maximum  likelihood  theory 
(Rao  (121),  while  the  results  for  0^  can  be  deduced  from  the  classical  results  using  the  strong 
Law  of  Large  Numbers  and  Central  limit  theorem  for  random  sum  involved,  namely,  as 
n  —  oo. 
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1  ^  d 


v~  L  "SJ  log  /(Uj;  )  —  0  in  probability 
1  ^ 


M  1 

N 

Z  ^  log/(M,;0)-i»  iV,(0,  a«-^). 

The  use  of  Lemma  1  then  allows  us  to  replace  the  random  scale  N4  bjr  the  usual  scale  n  to  get 
the  desired  properties  of  6  The  asymptotic  independence  of  fl  “  and  “  can  be  verified  via  the 
Cramer-Wold  device  as  in  the  proof  of  Theorem  I.  Details  are  omitted. 

Returning  now  to  the  full  likelihood  function  (17),  let  us  denote  by  and  <^„  the  likeli¬ 
hood  equation  estimators  based  on  it;  thus,  9„  and  satisfy  the  equations 

a 

I  log /(«/,«)  4- 
log  g(v,;  <<»)  -  0 


(21) 

a 

y  -2_  1 

1  Cfvf 

(22) 

y  1 
r  ^ 

where 

(23) 

H{x\9) 

log  (1  -  F(x;e)l. 

Comparing  (21)-(22)  with  (19)  it  is  seen  that  “  i^“,  while  differs  from  8".  We  seek  con¬ 
ditions  under  which  the  estimates  8„  and  8  “  are  asymptotically  equivalent.  For  the  purpose  of 
motivation  we  first  consider  the  important  special  case  of  Poisson  arrivals.  We  have  the  follow¬ 
ing. 

THEOREM  4;  For  the  MIGI\  queue  with  mean  interarrival  time  8  and  service  time  den¬ 
sity  g(v;  0)  we  have 

and 

(ii)  Vm  (8„  -  8)  and  -Jn  (8  “  —  8)  have  the  same  limit  distribution,  namely  /V,(0, 

PROOF;  (i)  We  have  /(m;  8)  «  8-'?-“''".  so  that  1  -  F(u\ 8)  -  e'"'”  and 
-  x9~\ 

The  first  of  the  Equations  (19)  reduces  to 


while  (21)  reduces  to 


-  T""  2:  «,  + 


Xn 


8^  f 


0. 


These  equations  yield  the  estimates  8"  and8„,  respectively,  as  given  by  (i). 
(ii)  In  view  of  Theorem  3  (ii)  it  suffices  to  show  that  as  n  —  «» 
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Now 


>/n  -  e  “)  —  0  in  probability. 


since  ►  17  in  probability  by  Lemma  I,  and  X„  has  a  limit  distribution  as  n  — •  «>. 

COROLLARY  I:  In  the  Ml M/X  queue  with  mean  interarrival  time  9  and  mean  service 
lime  <t>  we  have 

i  I  "■ 

-  —  X  V,,  and 

"  I 

(ii)  y/n  (ft „  —  9)  and  \fn  (9“-  9)  have  the  same  limit  distribution  namely  N^(0,  9^/ri). 

PROOF:  The  only  new  result  is  the  one  concerning  the  estimate  of  4^,  which  is  obtained 
from  (22). 

The  proof  of  Theorem  4  indicates  that  the  appropriate  condition  for  the  asymptotic 
equivalence  of  the  estimates  0,  and  "  is  that  as  n  — *  <» 

(24)  H(X„\9)  -*  0  in  probability. 

This  condition  is  satisfied  in  the  case  of  Erlangian  arrivals,  as  shown  in  the  following: 
EXAMPLE  1:  For  the  queue  EJGIX  we  have 


so  that 


and 


_  I). 


I  ~  f(u-9)  ~  T  —  Irf 

^  9 


It  follows  that 


and  so 


^  II  -  F(w;»)l  -  e  *"/* 
if 


H(u.9)  < 

9^ 


l(k-X)\. 


-jar  H(X„-,9)  <  —  0  in  probability 

yJn  9‘  yJn 


since  X„  converges  in  distribution  as  n 
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Under  the  condition  (24)  we  are  now  able  to  establish  the  desired  asymptotic  equivalence 
of  the  estimates#,,  and#,“.  We  have  the  following; 

THEOREM  5:  Under  the  condition  (24).  (#„  -  #)  and  Vn  («,“  -  #)  have  the  same 

limit  distribution. 


PROOF;  We  first  approximate 


7^  w  *>  ““  77  'k  *’ 

by  linear  functions  of  >/«(#„-#)  and  yfn  (#,"-#)  in  the  usual  manner  (Rao  [12]  and  then 
use  condition  (24),  namely. 


(24') 


1 


-Jj-  log  LJf.  g)  -  ^  log  L“{f.  g) 


0 


in  probability. 


4.  THE  M/M/1  QUEUE  -  m.l.e.’s  BASED  ON  A  SAMPLE 
FUNCTION  OBSERVED  OVER  A  FIXED  INTERVAL  (0,  tl 


In  Sections  2  and  3  we  observed  the  queueing  phenomenon  over  a  random  time  horizon 
(0.  D„),  where  D„  is  the  «th  departure  epoch.  This  choice  of  observation  period  resulted  in  a 
considerable  simplification  in  the  forms  of  the  estimates  and  their  limit  distributions.  Corollary 
1  shows  that  in  the  case  of  the  Ml  Ml  \  queue  the  moment  estimates  coincide  with  the  approxi- 
mat:  m.l.e.’s  of  the  interarrival  and  service  time  means,  these  latter  being  asymptotically 
equivalent  to  the  full  m.I.e.'s.  However,  due  to  the  simplicity  of  the  MIMI\  model  we  can 
consider  a  likelihood  function  based  on  a  continuous  observation  of  the  phenomenon  over  a 
fixed  interval  (0,  /]  and  study  the  asymptotic  properties  of  the  m.l.e.’s  as  t  —  «>.  This  latter 
sampling  plan  was  used  by  Wolff  [14]  who  derived  the  asymptotic  distributions  for  the  case 
p  <  1  directly  from  the  known  results  of  Billingsley  [31  for  Markov  processes.  In  this  section 
we  obtain  the  limit  distributions  of  the  m.l.e.’s  without  any  restriction  on  p,  using  elementary 
methods  similar  to  those  used  in  Section  2. 


Let  Q(i)  be  the  number  of  customers  present  in  the  system  (including  the  one  being 
served,  if  any)  at  time  t.  We  formulate  a  sample  function  representation  for  0(f)  as  follows. 
Let  i4(f)  be  the  number  of  arrivals  during  a  time-interval  (0,  f);  then  /4(f)  is  a  Poisson  pro¬ 
cess  with  parameter  X.  Let  D(t)  be  a  Poisson  process  with  parameter  fi  and  independent  of 
/4  (/).  Then  clearly 

(25)  0(f)  -  0(0)  +  A(f)-  dD(s) 

where  1,^  is  the  indicator  function  of  the  even  f/4.  Here  the  integral  represents  D(f),  the 
number  of  departures  from  the  system  during  (0.  fl;  thus, 

(26)  D(t)  -  f^' dD(s). 

The  total  time  during  (0,  f  J  that  the  server  was  busy  is  given  by 

(27)  B(f)  —  lp(j-)>o  ds- 
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The  interarrival  and  service  time  means  are  given  by  o  —  and  <>  —  respectively.  The 

traffic  intensity  is  p  -  X/fi.  Also,  we  denote  ^  -  mind,  p).  We  need  the  following  prelim¬ 
inary  results,  which  will  be  proved  in  the  appendix. 

LEMMA  2;  As  / ~ 


(i)  a.s. 


(ii)  -Mill.  — ►  ^  in  probability. 


Since  {0(r).  r  ^  0}  is  a  Markov  process  with  transition  intensities 

=  fl”'  (/■  >  0).  (/  >  1) 

it  follows  from  Billingsley  [3]  that  the  likelihood  function  based  on  the  sample  function  (QCs), 
0  <  s  <  /}  is  given  by 

(28)  L,(a.  b)  a  r®*'* 

The  m.l.e.’s  of  a  and  b  obtained  from  (28)  are  seen  to  be 


(29) 


a, 


A  it) 

We  have  the  following. 


and  ^ 


DU) 


THEOREM  6;  As  /  — 

(i)  a,  —  a  a.s.,  and  b,  —  b  in  probability, 


yTt  (a,  -  a) 

(  |o] 

o’  0  11 

yft  (b,  -  b) 

0  ()’/e)j 

PROOF:  (i)  By  the  strong  law  of  large  numbers  applied  to  A  (r)  we  obtain 
a,  —  k~'  —  a  a.s. 


Lemma  2  yields 

h  BU)/t 
DU)lt 


M 


in  probability. 


(ii)  We  write 

(30)  yTt  {a,  -  a) 


_  —  A  U)  X  t 


Here  AU)lt  —  k  a.s.  by  the  strong  Law  of  Large  Numbers.  Also,  as  /  — *  »  the  distribution  of 
A  it)  is  asymptotically  normal  with  mean  kt  and  variance  kt.  Therefore,  the  right  side  of  (30) 
converges  to  N^  (0,  a^)  in  distribution. 
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Next  consider 


(31) 


Dii)+\ 


(v,  -  b) 


yTt 


DU) 


(v;  -  b) 
DU) 


where  v,'  is  the  residual  service  time  of  the  customer  in  service  (if  any)  at  time  i.  Due  to  the 
lack  of  memory  property  of  the  exponential  density,  the  distribution  of  v/  is  free  of  i.  There¬ 
fore,  writing 

^  J  .  ^ 

DU)  ~  DU)  yTt 


and  using  Lemma  2(i)  we  see  that  the  second  term  on  the  right  side  of  (31)  converges  to  zero 
as  r  — '  oo.  The  first  term  can  be  written  as 


(32) 


4 


DU)  +  1 


D(i)+1 


DU)  si  DU)  +  1 


(V(  -  b). 


By  the  central  limit  theorem  applied  to  random  sums  we  obtain 


1 

VdoTm 


5((i+i 

I 

1 


(v,  -  b) 


N,(0.  b^). 


Again,  using  Lemma  2(i),  we  find  that  (32)  converges  in  distribution  to 


^  N^  (0.  b^)  -  N,(0,  bVf). 

Thus, 

(33)  V7(b,  -  b) /V,(0.  fcVC). 

The  asym£totic  independence  of  a,  and  6,  is  not  immediately  evident  because  of  the  depen¬ 
dence  of  Dit)  on  both  the  sequences  (u^)  and  (v/^).  However,  this  independence  can  be  esta¬ 
blished  as  in  the  proof  of  Theorem  1  (ii)  by  first  considering  the  random  variable 


(34) 


br- 


DU) 


Din 

Zv. 

1 


which  is  independent  of  a,,  since  the  processes  Ait)  and  Dit)  are  independent.  We  have 
■Jlib*-  b)  ^  N,(0,  b^)  as  before,  and 


(35) 


■Jl  ia,  -  a) 
Viib;-  b) 


Using  (35)  we  shall  prove  that 

(36)  aVr  (a,  -  o)  +  /SVt  ib,  -  b)  ^  N,(0.  +  P^b^i) 


for  arbitrary  real  numbers  a,  p.  The  desired  result  (ii)  will  then  follow  via  the  Cramir-Wold 
device. 
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Now  the  left  side  of  (36)  can  be  written  as 


(37) 


ay/l  (a,  —  a)  + 


+  /8 


■yTt  (b,  -b)- 


The  limit  distribution  of  ther  term  within  the  first  pair  of  brackets  in  (37)  is 
N\(0,  +  P^by^  on  account  of  (35).  The  term  within  the  second  pair  of  brackets  in  (37) 

can  be  written  as 


<»■  - ‘) + -v/?  (i  “  7^  ? 

■  im  ?  '  ‘*1 


(v,  -  b) 


where  we  have  used  the  fact  that  v,'  —  b  plays  the  same  role  as  v,  -  ft  (1  <  ;  <  f)(/))— see 
comments  following  (31).  Using  arguments  similar  to  the  ones  used  in  the  proof  of  Theorem 
l(i)  we  find  that  all  three  terms  in  the  last  expression  converges  to  zero  in  probability.  This 
completes  the  proof. 


5.  APPENDIX 


Only  the  outlines  of  the  proofs  are  given  below;  for  details  see  the  monograph  by  Prabhu 
Ill). 


Lemma  1  is  concerned  with  a  single  server  queue,  with  notations  as  in  Section  2,  while 
Lemma  2  is  concerned  with  the  Ml  MIX  queue  described  in  Section  4. 

PROOF  OF  LEMMA  1:  Let  XV„  be  the  nth  customer’s  waiting  time,  and  I„  the  idle  time 
(if  any)  preceding  this  customer.  Then 

(38)  =  max  (0,  W„  +  X„+,),  /„+,  -  -  min  (0,  IV„  +  X„+,) 

where  A”*  -  v*  -  u*(/f  >  1).  Let  So  =  0,  S„  -  +  J(2  +  •  •  •  +  X„  in  >  1).  Then  since 

H^o  =  as  has  been  assumed,  we  obtain  from  (38) 

(39)  w„-  S„+  X,  /„-/,  +  /j  -I-  -  min  S*. 

It  is  clear  that  the  successive  departure  epochs  D„  are  given  by 

D|  -  V,.  D„  -  u,  +  Hj  +  ...  +  +  W„_\  +v„(n  >  2). 

In  view  of  (39)  we  can  write 

(40)  D„  -  V|  +  Vj  +  . . .  +  v„  +  (n  >  I). 

Now  it  is  known  that  (i)  if  p  >  I,  then  — *  /a.s.,  while  (ii)  if  p  <  1,  a.s.,  but  n"’ 

-*  max(0,  a  -  b).  Therefore, 

,4„  A  .  ^  ^ 

n  n  n 

—  6  +  max  (0,  a  —  b)  ^  max  (a,  b). 
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N^ext  let  N^{i)  —  maxU:  Uj  +  uj  +  ...  +  m*  <  /).  Then  by  the  elementary  renewal  theorem, 
r'N^(t)  — »  a"’  a.s.  Since  —  «»  by  (41),  we  obtain 


NAiPn) 

D, 


— *  a  '  a.s. 


From  (41)  and  (42)  it  follows  that 


Na  _  Na(DJ  D„  . 


- -  a  '  max  (a,  ft)  -  w 

n 


as  required. 


PROOF  OF  LEMMA  2:  (i)  We  can  write 


where 


Z)(/)-f)(/)- 

/  lo(i-).o  dD{s)  -  -  m{t)  -  -  inf  \A  (s)  -  D(s)l 


and  it  is  known  that 

-m(t) 


—  max  (0,  /A  -  X)  a.s. 


Therefore, 


DU)  DU)  .  mU)  _ /A 

— ; —  * - + - -  (I  —  max  (0,  /A  —  X)  “  MC  » 


(ii)  We  can  write  B(t)  -  t  -  /(/),  where  lU)  is  the  idel  time  during  (0,  fl  and  it  is  known 
that 

(AO\  lU) _  ^  /A  1 


Therefore, 


max  (0.  1  -  p)  in  probability. 


-  1  -  —  -  1  -  max(0,  1  -  p)  - 
It  > 
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ABSTRACT 


A  moihod  is  dcvciopod  for  deiormming  ihc  opiimal  policy  for  entry  of  tus- 
lonicrs  from  many  indepcndcni  classes  of  I'oisson  arrivals  o  a  lirsi-como. 
lirsi-serve  (for  cusiomcrs  admiiied  lo  ihc  gucuel  single-server  queue  with  e\- 
poneiilial  service  limes  The  soluiion  lechnigue  uiili/cs  a  senii-Vlarkov  formu- 
laiion  of  llic  decision  problem 

INTRODUCTION 

A  small  but  substantial  collection  of  literature  has  developed  on  optimal  control  of  entry 
to  a  queue  starting  with  Naor's  work  [lOl  in  1969  and  continuing  to  the  present.  Most  of  this 
work  has  been  concerned  with  optimal  policies  and  their  properties  for  queues  serving  a  single 
class  of  customers.  In  these  models  the  reward  and  cost  structure  is  the  same  for  each  custo¬ 
mer  although  a  customer's  reward  may  be  the  value  of  a  random  variable. 

In  this  paper,  a  model  is  considered  in  which  several  classes  of  customers  arrive  in 
independent  Poisson  streams.  Each  class  has  its  own  reward  structure  in  that  each  member  of 
class  nt  receives  a  reward  R„  for  service  and  pays  C,„  per  unit  time  spent  in  the  system.  This 
generalization  has  obvious  applications  in  queueing  systems.  For  example,  consider  an  airport 
serving  many  classes  of  commercial  aircraft  (jumbo  Jets,  wide-bodies,  four  engine  jets,  three 
engine  jets,  two  engine  jets)  in  which  each  class  has  its  own  reward  for  service  and  cost  of  wait¬ 
ing.  It  does  not  appear  that  the  optimal  control  policy  for  such  cases  has  been  determined. 

Two  types  of  optimum  control  policies,  the  individual  optimum  and  the  social  optimum, 
are  considered  in  this  paper.  For  an  individual  optimum  policy,  join  or  balk  decisions  are  based 
on  the  expected  gain  of  each  individual  customer  at  the  time  of  his  arrival.  If  his  expected  gain 
is  nonnegative  he  joins;  otherwise  he  balks.  Under  a  social  optimum  policy,  the  join  or  balk 
decision  is  made  based  on  maximizing  the  expected  gain  per  unit  time  considering  arrivals  from 
all  classes  of  customers. 
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A  review  of  the  literature  indicates  that  ‘‘control-limit”  policies  are  optimal  for  a  single 
class  of  customers.  This  type  of  policy  specihes  that  a  customer  be  admitted  if  the  number  of 
customers  already  in  the  system  is  less  than  or  equal  to  a  given  threshold  value  (the  "control 
limit”).  If  the  control  limit  is  exceeded,  the  customer  is  turned  away.  Naor  assumed  that  a 
control-limit  policy  is  optimal  for  a  single  class  of  customers  and  then  compared  the  control 
limits  for  individual  and  social  optima.  Yechiali  113]  proved  that  a  control-limit  policy  is  indeed 
individually  and  socially  optimum  for  a  GIIMI\  queueing  system  serving  a  single  class  of  cus¬ 
tomers.  Subsequent  work  by  several  authors  Tor  example,  see  (61,  17),  18],  (9),  (12),  and 
[14])  demonstrated  that  control-limit  policies  £.re  optimal  for  single  class  models  under  even 
more  general  assumptions. 

When  considering  several  classes  of  customers,  the  argument  that  the  individual  optimum 
policy  is  a  control-limit  policy  carries  over  directly,  as  shown  later,  from  the  argument  for  a  sin¬ 
gle  class  model.  However,  it  is  not  possible  to  carry  over  Yechiali's  argument  for  the  social 
optimum  policy  and  a  different  approach  is  used. 

Previous  work  with  several  classes  of  customers  appears  to  be  quite  limited.  Balachandran 
and  Schaefer  [I,  2]  deal  with  several  classes  of  arrivals  to  MlGl\  queues.  However,  each  class 
has  the  ability  to  adjust  its  arrival  rate  so  that  the  class  with  the  most  favorable  net  benefit 
(reward  minus  expected  cost)  dominates  the  arrivals  to  the  exclusion  of  all  other  classes  of  cus¬ 
tomers.  Furthermore,  this  work  is  based  entirely  on  long  term  expected  values  and  no  custo¬ 
mers  are  able  to  take  advantage  of  short-lived  phenomena,  e.g..  a  customer  who  does  not 
belong  to  the  single  class  which  dominates  arriving  to  find  the  system  empty  will  be  denied  ser¬ 
vice.  Edelson  and  Hildebrand  (4|  come  closer  to  the  problem  dealt  with  in  this  paper.  They 
consider  the  Ml  MIX  model  for  classes  with  different  reward  and  cost  parameters  and  balking. 
The  claim  to  have  considered  more  than  two  classes  of  customers,  but  no  results  for  more  than 
two  classes  are  presented,  and  in  what  seems  to  be  a  contradiction  they  indicate  in  their  conclu¬ 
sion  that  the  solution  procedure  they  have  used  (which  is  not  specified  in  their  paper)  does  not 
extend  to  cases  where  "dichotomous  classification  of  customers  is  not  acceptable.”  While  they 
are  interested  in  comparing  revenue  maximizing  policies  with  the  social  optimum,  it  should  be 
noted  that  the  solution  given  for  the  two  class  social  optimum  policy  in  their  first  example,  after 
appropriate  adjustments  for  different  definitions.,  was  checked  and  found  to  yield  the  same  pol¬ 
icy  that  would  be  obtained  by  the  techniques  established  in  this  paper.  Furthermore,  this  check 
provides  an  indication  that  the  general  techniques  set  forth  in  this  paper  are,  in  addition  to 
being  workable,  correct. 

In  this  paper,  after  defining  the  model,  it  will  be  shown  that  Naor's  solution  for  the  indi¬ 
vidual  optimum  for  a  single  class  of  customers  easily  extends  to  M  classes  of  customers.  The 
extension  of  his  social  optimum  solution  to  many  classes  of  customers  is  then  made  by  adapting 
the  semi-Markov  decision  process  technique  used  by  Yechiali  for  a  single  class  of  customers. 
The  optimal  policy  is  shown  to  be  a  control-limit  policy  and  simple  bounds  on  this  policy  are 
presented.  An  example  of  the  application  of  the  technique  to  three  classes  of  customers  is 
given. 

THE  MODEL 

A/ classes  of  customers  are  considered.  Customers  of  class  m,  m  -  1,  2 . A/,  arrive 

in  a  Poisson  stream  with  mean  rate  The  service  times  of  the  single  server  are  independent, 
identically  distributed,  exponential  random  variables  with  mean  I/^.  Thus,  the  model 
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represents  an  Ml  MIX  queueing  system  with  customers  Trom  M  separate  groups  or  classes. 
Each  class  has  its  own  reward  and  cost  structure.  Once  a  customer  enters  the  system,  service  is 
hrst-come,  first-served,  regardless  of  class.  A  member  of  class  m  receives  a  reward  for  service, 
and  pays  C„  per  unit  time  spent  in  the  system.  R„  is  required  to  be  greater  than  or  equal 
to  C,h/m  for  all  m.  (Customers  from  any  class  for  which  this  is  not  true  would  balk  under  all 
circumstances.)  For  determining  the  individual  optimal  policy,  each  customer  is  assumed  to  be 
able  to  determine  the  number  of  customers  in  the  system  at  the  time  of  his  arrival.  Also,  for 
determining  the  social  optimal  policy,  an  administrator  is  assumed  who  knows  the  number  of 
customers  in  the  system  at  the  time  of  each  arrival.  An  optimal  policy  is  sought;  that  is,  a  set 
of  join  or  balk  decisions  is  sought  for  each  class  for  each  possible  state  (the  total  number  of 
customers  in  the  system)  to  maximize  (i)  the  expected  return  to  the  self-optimizing  (indivi¬ 
dual)  customer  or  (ii)  the  gain  per  unit  time  of  all  customers  (social  optimum) . 

SOLUTION  FOR  THE  INDIVIDUAL  OPTIMUM 

If  an  arrival  from  class  m  finds  /  customers  ahead  of  him,  his  expected  net  benefit  for 
joining  the  system  is  R„  ~  (i  +  1)  C„lfi.  Since  the  net  benefit  for  balking  is  zero,  the  custo¬ 
mer  will  join  the  system  if  the  number  of  customers  in  the  system  is  less  than  where  is 
such  that 

(1)  R„-  +  X)CJix  <  0  ^  R„-  CJfi. 

(Customers  are  assumed  to  join  the  system  if  their  expected  net  benefit  for  joining  is  zero.) 
Thus,  self-optimizing  customers  of  class  m  determine  a  balking  point  such  that 

(2)  1/?,„m/C„1. 

where  the  brackets  indicate  the  greatest  integer  function.  (2)  is  a  straight-forward  extension  of 
Naor’s  result  for  a  single  class  of  customers. 

SOLUTION  FOR  THE  SOCIAL  OPTIMUM 

For  the  social  optimum  problem,  an  administrator  decides  whether  or  not  a  customer  of 
class  m  can  join  when  i  customers  are  ahead  of  him.  The  sum  of  the  expected  net  benefits  per 
unit  time  of  all  arrivals  of  all  classes  is  shown  to  be  maximized  by  a  policy  that  imposes  a  vector 

«o  of  forced  balking  points  on  the  customers,  ran”  (no,-  "Oj . ®  member  of 

class  m  is  allowed  to  join  if  the  state  of  the  system  is  less  than  but  must  balk  otherwise. 

Let  n  -  (w,.  ^2 . rtnf)  be  a  vector  of  balking  points.  The  sum  of  the  expected  net 

benefits  per  unit  time  of  all  arrivals  when  the  balking  points  given  by  n  are  chosen,  is  given  by 

(3)  gin)  -  i  x;  (n)  R„-  £  C„  LJn). 

m^\  m—l 

X,^  (n)  and  L„in)  are,  respectively,  the  effective  arrival  rate  for  class  mand  the  contribution  of 
class  m  to  the  expected  number  of  customers  in  the  system  when  balking  points  n  are 
employed.  K'„  (n)  and  L„  in)  depend  on  n  because  the  stationary  probability  that  /  customers 
are  in  the  system,  <l>iin),  depends  on  n. 

Rue  [11]  shows  that  the  following  semi-Markov  decision  process  formulation  of  gin)  is 
equivalent  to  (3); 
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(4)  gin)  -  £  k„ii)k„{R„  -  a  +  \)  CJn\, 

/“O  m*  I 

where  n*  —  max  4>iin)  is  the  stationary  probability  that  /  customers  are  in  the  system 

m 

given  that  the  balking  points  n  are  employed,  and 

1  if  /  <  n„, 

'  0  if  I  >  /,„ 

(It  may  be  helpful  to  note  at  this  point  that  the  term  ~  H  +  is  the  gain  rale  in 

state  i  due  to  class  m  customers  joining  the  queue.)  The  work  of  Derman  (3]  can  be  used  to 
limit  the  semi-Markov  decision  process  formulation  to  nonrandomized  policies  as  in  (4).  Policy 
iteration  can  be  used  to  obtain  from  (4)  the  maximum  gain  rate  and  socially  optimal  joining 
policy. 

Theorem  1  justifies  restricting  the  consideration  of  optimal  policies  to  control-limit  poli¬ 
cies  as  in  (3).  However,  lemma  1  must  be  proved  first. 

LEMMA  I;  The  social  optimum  policy  will  not  allow  reneging. 

PROOF;  Suppose  customer  A  of  class  /»  arrives  at  time  and  joins  the  system.  Later, 
at  time  T^,  customer  A  departs  the  system  before  he  is  served.  The  actions  of  customer  A 
affect  no  customer  who  arrived  before  him.  The  contribution  of  customer  A  to  the  net  gain  of 
the  system  over  the  interval  IT^,  Tp]  is  -C^iTp  -  T^)  <  0,  his  holding  cost  for  the  lime  he 
is  in  the  system.  If  no  other  customer  arrives  before  customer  A  departs,  his  actions  have  no 
effect  on  those  customers  arriving  after  him.  If  other  customers  arrive  during  Tp],  the 
only  effect  of  customer  A's  temporary  presence  in  the  system  is  to  possibly  cause  a  customer  to 
balk  when,  in  fact,  without  the  presence  of  A  this  customer  would  be  profitable  (and  thus  join 
(lie  system).  Thus,  the  contribution  of  customer  A  to  the  system  is  negative  when  compared 
with  his  not  joining  the  system  at  all. 

THEOREM  1;  A  control-limit  policy  is  optimal  for  each  class. 

PROOF;  Suppose  that  there  exists  a  class  m  and  a  state  i  such  that  in  the  optimal  policy, 
p,  k„ii)  -  1,  but  k„ii  -  1)  “  0;  that  is,  the  optimal  policy  for  class  m  is  not  a  control-limit 
policy.  The  optimal  gain  rate  is  denoted  by  g*.  If  the  stationary  probability  of  the  system  occu¬ 
pying  state  /  under  policy  p,  is  equal  to  zero,  then  a  policy  which  is  the  same  as  the  optimal 
except  that  k„(i)  —  0  also  yields  g*.  To  prove  Theorem  I  for  the  case  ^  0,  consider  the 
following  modification  to  the  model.  At  the  completion  of  a  service,  each  customer  returns  his 
expected  net  benefit  for  recalculation  based  on  the  current  state  of  the  system.  Likewise,  his 
occupation  costs  incurred  during  the  service  just  completed  are  also  returned.  The  customers 
remain  in  the  order  they  arrived,  but  the  administrator  recomputes  the  expected  net  benefit  of 
each  customer  based  on  the  number  now  ahead  of  that  customer  and  uses  the  given  policy,  p, 
to  decide  whether  or  not  each  customer  can  re-enter  (stay  in)  the  system.  Because  there  is  no 
discounting,  the  gain  rate  of  this  model  is  equivalent  to  that  of  the  original  model  for  the  same 
policy.  Since  the  transitions  are  Markovian,  all  relevant  information  about  the  future  of  the 
system  is  contained  in  the  current  state  of  the  system.  (The  state  of  the  system  must  give  the 
position  of  every  customer  to  accommodate  the  new  method  of  assessing  costs.)  A  customer  of 
class  m  who  had  joined  when  /  customers  were  ahead  of  him  would  be  forced  to  depart  when 
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the  number  of  customers  ahead  of  him  dropped  to  i  -  1,  since  k„{i  -  1)  -  0.  Thus,  the  cus¬ 
tomer  would  be  forced  to  renege.  By  Lemma  1,  the  given  policy  cannot  be  optimal  for  the 
modified  or  original  model. 

Since  the  expected  net  benefit  for  members  of  any  class  Joining  the  system  decreases  as 
the  number  of  customers  in  the  system  increases,  the  optimality  of  a  control  limit  policy  is  a 
reasonable  result. 

It  is  no,"  shown  (Theorem  2)  that  for  each  class  m,  wo,  <  ,  where  is  the  socially 

optimal  balking  point  for  class  m  customers  and  n,  is  the  individual  optimum  balking  point  for 
class  m.  Thus,  n*-  max  {n,  )  can  serve  as  a  bound  on  the  state  space  required  for  the  policy 

m 

iteration  solution  of  (4). 

THEOREM  2.  For  each  class  m,  n,)^  < 

PROOF:  First,  the  decision  made  by  the  administrator  for  a  given  arrival  does  not  affect 
the  time  of  arrival  of  any  customer  yet  to  arrive.  As  previously  shown,  satisfies  (1) 

+  1)  Cjti  <  0  <  R„-  cjfi. 

Suppose  a  customer,  customer  A,  of  class  m  arrives  to  find  the  state  of  the  system  i  >  Let 
be  the  time  of  arrival  of  customer  A,  and  Tg  be  the  time  of  arrival  of  the  next  customer, 
customer  B.  In  view  of  (1),  let  a  <  0  be  the  expected  net  benefit  of  customer  A  joining  the 
system.  If  all  expected  costs  and  rewards  are  assigned  to  a  customer  upon  arrival  (as  in  (4)), 
the  contribution  of  the  interval  IT^.  Tg\  to  the  expected  net  gain  is  a  <  0  if  customer  A  is 
allowed  to  join.  The  decision  regarding  customer  A  does  not  affect  the  expected  net  gain  of 
customers  who  arrived  before  him,  but  it  does  affect  the  expected  net  gain  of  those  arriving 
after  him.  If  customer  A  does  not  join,  the  state  of  the  system  found  by  customer  B  and  all 
others  after  him  is  less  than  or  equal  to  the  state  of  the  system  if  customer  A  joins.  Since 
-  a  +  \  )  the  expected  gain  for  a  customer  of  class  m  joining  when  i  are  in  the  sys¬ 

tem,  is  a  strictly  decreasing  function  of  /,  the  contribution  to  the  expected  net  gain  of  the  inter¬ 
val  [Tg,  Tf  1  for  any  T(  >  Tg  is  at  least  as  large  when  customer  A  balks  as  when  he  joins. 
Thus,  if  t  >  n,^,  forcing  a  customer  of  class  m  to  balk  yields  a  larger  expected  net  gain  or  gain 
rate  than  allowing  him  to  join. 

The  two  theorems  proved  above  make  it  possible  to  solve  for  the  social  optimum  policy 
using  policy  iteration  or  linear  programming.  For  A/ classes  of  customers,  we  seek  to  determine 
the  n  which  maximizes  the  gain  rate  given  in  Equation  (4).  The  solution  technique  will  be 
illustrated  by  an  example.  Suppose  that  three  classes  of  customers  arrive  at  a  single  exponential 
server  who  has  a  service  rate  capability  of  four  customers  per  unit  time.  The  reward  for  ser¬ 
vice,  cost  per  unit  time  in  the  system,  and  mean  arrival  rate  of  each  class  are  given  in  Table  1. 

TABLE  I  —  Model  Parameters  for  the 
Three-Class  Example 


Class 

Cg, 

k  m 

1 

3 

4 

2 

2 

2 

3 

4 

3 

4 

7 

6 
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To  see  what  g(n)  looks  like  for  a  particular  policy  n,  consider  g{n,)  where  n,  is  the  indi¬ 
vidual  optimum  policy.  From  (2) 

“  ("v,.  "sj.  ”v,)  “  (3,  2.  2). 

Since  max  (n,  )  -  3  at  most  states  0  through  3  are  required  to  evaluate  g(n)  for  any  «, 

m  ^ 

e.g., 

gO,  2.  2)  =  g(n,)  -  l/f|  -  (/  +  1)C,/m) 

i-O 

+  L  X„  l/?„  -  (/ -I-  1)C„/m1 

("0  m*2 

where  the  0,(n,),  i  -  0.  1,  2,  3  can  be  obtained  from  the  simultaneous  solution  of  the  three 
rate  equations 

.1 

m-  1 

3  3  _ 

3 

(X|  -E^)<^2(/j,)  =  ^  +!*■•{> 

m-  1 

and  the  normalizing  equation 

X  -  1. 

1-0 

The  first  iteration  of  Howard's  [S]  policy  iteration  algorithm  yields  the  individually  optimal 
policy,  n,.  Because  the  relative  values  of  occupying  the  various  states  (Howard’s  v,^)  are  all  set 
to  zero,  a  social  optimizing  customer  cannot  determine  the  effect  his  joining  has  on  later  arriv¬ 
ing  customers  and,  thus,  can  do  no  better  than  the  individually  optimal  policy.  Thus,  the  first 
iteration  yields  a  control-limit  policy  with  balking  points  n  =■  (3,  2,  2)  and  the  expected  gain 
rate  associated  with  the  policy  is  g  2.486.  As  the  policy  iteration  algorithm  proceeds,  better 
and  better  knowledge  of  the  v,'s  will  result  in  better  and  better  policies  until  an  optimal  policy  is 
found. 

The  second  iteration  of  Howard’s  algorithm  leads  to  another  control-limit  policy  with 
balking  points  n  »  (2,  0,  I)  and  an  expected  gain  rate  of  ^  S.375.  This  policy  excludes  class 

two  from  the  system. 

The  socially  optimal  policy  is  found  on  the  third  iteration  and  is,  of  course,  a  control-limit 
policy.  The  balking  points  are  no  -  (1,  0.  I)  and  the  expected  gain  rate  is  S.833.  Again,  class 
two  is  excluded  from  the  system.  The  stationary  probabilities  of  the  system  occupying  the  vari¬ 
ous  states  under  the  optimal  policy  are  <^<no)  *•  (1/3.  2/3,  0,  0>. 

SUMMARY 

The  single  class  of  customers  model  of  Naor  is  extended  in  this  paper  to  several  customer 
classes  and  then  solved  for  the  individual  optimum  policy  and  the  social  optimum  policy  and 
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associated  gain  rate.  Both  the  individual  and  social  optimum  policies  are  shown  to  be  control- 
limit  policies.  In  addition,  the  socially  optimal  control  limit  for  each  class  is  shown  to  be  no 
greater  than  the  individually  optimal  control  limit  for  the  same  class.  The  solution  techniques 
are  illustrated  by  an  example. 
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ABSTRACT 

An  .V//67I  queueing  •ivsiem  is  studied  in  wliieh  the  service  lime  required 
by  a  customer  is  deirendem  on  the  inierarrival  lime  between  his  arrival  and 
that  of  his  predecessor  Assuming  the  two  variables  are  "associated."  we  prove 
that  ilic  expected  delay  in  this  system  is  less  than  or  equal  to  that  of  a  conven¬ 
tional  MU1I\  queue  This  conclusion  has  been  verilicd  via  simulation  by 
Mitchell  and  I’aulson  Ifl  for  a  special  class  of  de|>cndcni  Ml Ml\  queue  Their 
model  is  a  special  case  of  the  one  we  consider  here  We  also  study  another 
modified  07/071  queue,  where  the  arrival  process  and/or  the  service  priHtcss 
are  individually  "associated" 


1.  INTRODUCTION 

The  conventional  C//C/I  queueing  model  considered  in  most  papers  in  the  literature 
assumes  that  ttic  .sequences  of  interarrival  times  and  service  times  are  i.i.d.  (independent  and 
identically  distributed)  random  variables.  However,  the  independence  assumption  may  not  be 
realistic  for  many  real  world  problems.  The  purpose  of  this  note  is  to  investigate  the  effect  on 
average  delay  (queueing  time  only)  of  customers  in  systems  whose  inierarrival  and  service 
times  are  associated,  a  concept  of  positive  dependence  developed  by  Esary,  Proschan  and 
Walkup  [6|. 

In  a  recent  paper,  Mitchell  and  Paulson  [9]  studied  via  simulation  an  MlMI\  queue  with 
the  modification  that  a  customer's  service  time  and  the  interarrival  time  between  his  arrival  and 
that  of  his  predecessor  are  positively  correlated  random  variables  having  a  bivariate  exponential 
distribution.  Their  simulation  results  indicate  that  this  type  of  dependency  reduces  the  mean 
waiting  time  of  customers  as  compared  to  the  usual  M/ M/\  queue.  Some  related  results  also 
appeared  in  Conolly  (2],  Conolly  and  Hadidi  (3,  4).  Motivated  by  Mitchell  and  Paulson's  simu¬ 
lation  results,  we  shall  show  here  in  Section  3  that  their  conclusion  can  be  proven  analytically 
under  weaker  conditions.  In  Section  2,  we  briefly  summarize  the  useful  concept  of  association 
of  random  variables.  Another  variant  of  C//C/1  queues  with  dependent  interarrival  times  or 
service  times  will  also  be  discussed  in  Section  4. 

2.  ASSOCIATION  OF  RANDOM  VARIABLES 

The  concept  of  association  of  random  variables  was  developed  by  Esary,  Proschan  and 
Walkup  [6].  It  is  a  very  useful  tool  in  the  study  of  reliability  systems  (see  Barlow  and  Proschan 
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(ll,  Esary  and  Proschan  (51),  queueing  systems  (see  Niu  [10]),  and  simulation  (see  Heidel- 
berger  and  Iglehart  (71). 

DEFINITION  1:  Random  variables  (AT,.  Af,)  are  called  associated  if  cov 
[/(AT),  if  (Af)l  >  0  for  all  pairs  of  nondecreasing  functions  /and  g. 

The  following  is  a  partial  list  of  properties  enjoyed  by  associated  random  variables: 


(PI): The  set  consisting  of  a  single  random  variable  is  associated. 

(P2);  Nondecreasing  functions  of  associated  random  variables  are  associated. 

(P3);  Any  subset  of  associated  random  variables  are  associated. 

(P4):lf  two  sets  of  associated  random  variables  are  independent  of  each  other,  then  their 
union  is  a  set  of  associated  random  variables. 


(P5):  A  set  of  independent  random  variables  are  associated. 


The  last  property,  P5,  is  a  direct  consequence  of  PI  and  P4.  As  an  illustration,  we  will 
next  show  that  the  bivariate  exponential  distribution  discussed  in  Mitchell  and  Paulson  [9]  is 
generated  from  a  pair  of  associated  random  variables.  Consider  the  random  vector  (AC,  Y) 
defined  by 


(AT.  Y) 


I  i 


,-i  /-I 


where  X,,  /  «  1,  2,  ...  ,  are  i.i.d.  random  variables,  K,,  /  =  I,  2 . are  i.i.d.  random  vari 

ables,  and  N  is  an  integer-valued  positive  random  variable. 


PROPOSITION):  (Af,  P)  is  associated. 


PROOF.  Let  ,/  and  g  be  an  arbitrary  pair  of  nondecreasing  functions.  Conditioning  on 
A  “  n,  we  have 


cov  [/(AT. 

(  1 

Y),  giX,  Y)] 

in  1  {  '^1 

—  £  jcov  1 

(  f 

in  "1  ll"  " 

+  cov|£j/ 

I  at,.  I  K,  lA-  «1,  dg  £  AT,.  I  Y, 
(i-i  '-1  I  1  I'-i  '-I 

|A  -  n 

Now,  given  N  ^  n,  the  vector  (AT), 

Irt  n 

X,,  ^  Y, 

(-1  (-1 

ated.  Therefore,  the  first  term  above  is  nonnegative.  The  conclusion  of  the  proposition  then 


.  X„,  T, . Y„)  is  a  collection  of  independent 

is  associ- 


follows  by  observing  that  both 


t  K,  I A  -  «  and  £  g  i  AT,,  f  Y, 

((-I  /-I  i-l  I- 1 


lA 


are 


nondecreasing  functions  of  n. 
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REMARK:  When  both  X,,  i  =  1.  2 . and  K,,  /  =  1,  2 . have  exponential  distri¬ 

butions  and  N  is  geometric,  then  (X,  Y)  has  the  bivariate  exponential  distribution  described  in 
[9], 

3.  AN  M/G/l  QUEUE  WITH  DEPENDENT  ARRIVAL  AND  SERVICE 

Consider  the  following  model  of  a  single  server  queue  with  infinite  waiting  room.  Denote 
by  T„  the  interarrival  time  between  customers  C„  and  C„+|  with  E(T„)  -  l/K.  Let  S„  be  the 
service  time  of  customer  C„  with  £(S„)  —  \/fi  and  k/ft  <  1.  We  assume  (r„.  S„+|), 

n  =  0,  1 . is  a  sequence  of  i.i.d.  random  vectors  and  (T„,  S„+|)  is  associated  for  each  n.  It 

should  be  noted  that  this  model  includes  both  the  conventional  GllGlX  queue  (where  T„  and 
5„  +  |  are  independent)  and  the  MlMl\  queue  considered  by  Mitchell  and  Paulson  (91  by  letting 
(r„,  S„  +  |)  to  have  their  bivariate  exponential  distribution  (see  remark  after  Proposition  1). 

Denoting  the  delay  of  customer  C„  by  /)„,  it  is  well-known  that 

(1)  =  maxlO.  D„  +  -  T^l, 

or  equivalently, 

(2)  D„,,-  A„=  D„  + S„-  T,. 

where  =  -  min  (0,  D„  +  S„  -  T„\.  We  will  assume  that  the  first  customer  arrives  at  time  0 
and  the  system  is  initially  empty,  i.e.,  O,  =  0.  The  following  key  lemma  will  be  needed. 

LEMMA  1:  cov  [D„.  S„1  <  0  for  all  n  >  1. 

PROOF:  The  assertion  is  clearly  true  for  n  “  I  since  £)|  =  0.  For  n  ^  2,  (7'„_|,  S„)  is. 
by  assumption,  associated  and  independent  of  -(D„-i  +  S„-|).  Therefore, 
(-/)„-!  -  S„-|.  r,,,.  S„)  is  associated  by  P4.  Observe  that  -D„  and  S„  are  nondecreasing 
functions  of  S„  |,  r„  i.  S„).  Hence,  by  P2,  {-D„,  S„)  is  associated  and  conse¬ 
quently  cov  ID„,  S„1  <0.  n  P  n 


Now,  squaring  both  sides  of  (2)  and  taking  expectations,  we  have 

(3)  EiD„K^)  +  E{A,})=  EiD,})  +  E(S„  -  T„)^  +  2E{D„S„)  -  2E[D„T„). 


where  we  have  used  the  fact  that  D„,^A„  =  0.  Letting  n  go  to  infinity  in  (3),  we  can  cancel 
E{D^\^ )  and  EiD,;)  from  both  sides  assuming  they  are  finite.  Since  D„  and  T„  are  indepen¬ 
dent  and,  by  Lemma  1,  cov  [D„,  S„)  =  E(D„S„)  -  E(D„)  E{S„)  ^  0,  the  right-hand-side  of 
(3)  is  less  than  or  equal  to 

E(S„  -  +  2  E(D„)  E(S„)  -  2  E(D„)  £(  T„). 


Noting  that  E(T„  -  S„)  =  E(A„),  (3)  simplifies  to 


(4) 


£(D„)  < 


E(T„-  s„y 

2E{T„  -  S„) 


EiA^) 
2E(A„)  ■ 


We  are  now  ready  for  the  main  result  in  this  section. 

THEOREM  1:  If  the  arrival  process  is  Poisson  and  the  system  is  in  equilibrium,  then 
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S.  Nil) 


(5) 


E(D„)  < 


X  £(S,f) 

2(1  -  X£(S„))  ■ 


where  the  righi-hand-side  term  is  the  expected  delay  of  a  conventional  A//C/1  queue. 


PROOF:  Note  that  (A„\A„  >  0)  is  the  idle  period  ended  by  the  arrival  of  customer  C„+i. 
Therefore,  by  the  memoryless  property  of  exponential  distributions,  (A„\A„  >  0)  is  also 
exponential  with  rate  X.  Hence, 

E(A^)  E{A^,\A„  >  0)  •  P{A„  >  01 

2E(.A„)  “  2E(A„\A„  >  0)  •  P{A„  >  0) 

_  E{A„^A„  >  0) 

2EU„\A„  >  0) 


=  1/X 


and  from  which  (5)  follows. 


Q.E.D. 


For  arrival  processes  other  than  Poisson,  the  analysis  of  (4)  becomes  more  difficult 
because  we  do  not  know  the  distribution  of  (AjA„  >  0).  However,  various  bounds  for  it  may 
be  obtained  by  considering  special  classes  of  arrival  processes  (see  Marshall  [8]).  We  shall  not 
pursue  this  further  here  except  for  mentioning  that  all  upper  bounds  for  expected  delay  in  spe¬ 
cial  classes  of  GI/G/l  queues  obtained  by  Marshall  18]  can  also  be  applied  to  our  modified 
GI/G/l  queues. 

4.  SOME  RELATED  RESULTS 


In  this  section,  we  will  consider  another  modification  of  GllGl\  queues  which  is  different 
from  the  one  discussed  in  Section  3.  We  shall  assume  that  the  sequences  of  service  times  and 
interarrival  times,  1S„,  n  ^  l)  and  {T„,  n  ^  1),  are  identically  distributed  and  associated,  i.e., 

(S|,  S2,  ,  S„)  and  (Tj,  Tj . T„)  are  associated  vectors  for  all  n  >  1.  Of  course,  this  is 

again  a  generalization  of  the  conventional  GI/G/\  queue  because  independent  random  vari¬ 
ables  are  associated.  Now,  it  is  easy  to  see  that 


D„ 


max 


0.  S,_,  -  r„. 


n  *  I 

I 


/-I 


Hence,  {D„.  S„)  and  (D„,  -T„)  are  associated  vectors  for  all  n  ^  1.  It  follows  that 
E{D„  S„)  >  E{D„)  £(S„)  and  E(D„T„)  <  E(D„)  E(T„).  An  argument  similar  to  Section  3 
will  then  lead  to 


(6) 


£(D,)  > 


E(T,  - 
2£(r,-5,) 


2E(AJ- 


Again,  the  second  term  on  the  right-hand-side  of  (6)  is  difficult  to  analyze.  However,  we 

have 


THEOREM  2;  If  |r„,  n  ^  ll  are  i.i.d.  exponential  random  variables,  i.e.,  the  arrival  pro¬ 
cess  is  Poisson,  then,  under  stationary  conditions,  we  have 
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£(D„)  > 


A  £(s„-) 

2(1  -  A£(S„)) 


In  other  words,  associated  service  times  in  an  M/C/1  queue  tend  to  increase  the  expected 
delay  of  a  customer.  This  is  a  somewhat  expected  result. 

When  |r„.  n  ^  U  is  a  renewal  process  (not  necessarily  Poisson),  lower  bounds  for 
£(D„)  may  be  found  through  (6)  for  several  special  classes  of  interarrival  times  as  in  Marshall 
18|.  As  a  final  remark,  we  note  that  the  departure  process  of  a  conventional  C//D/1  queue 
(the  service  times  are  deterministic)  is  an  associated  process.  So,  (6)  may  be  used  to  find  a 
lower  bound  for  the  expected  delay  of  a  subsequent  station. 
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ABSTRACT 

In  ihe  (eniporarily  isolated  situation  in  which  a  warship  linds  itself  during  a 
mission,  not  only  spare  parts,  but  also  "spare"  crewmen  in  various  categories  of 
specialization  must  be  on  board. 

Mathematical  models  for  the  probabilities  of  mission  survival  for  personnel  and 
for  personnel  and  materiel  jointly  are  proposed.  A  practical  example  is  worked 
out:  the  optimal  allocation  of  spare  crewmen  to  difl'erent  categories  of  speciali¬ 
zation  is  calculated. 


1.  INTRODUCTION 

In  relation  to  a  system  on  board  a  warship  (weapons,  communications  etc.)  a  meaningful 
figure  of  merit  is  the  mission  reliability,  defined  as:  "the  probability  that  the  system  will  operate 
in  the  mode  for  which  it  was  designed  for  the  duration  of  the  mission"  [ll.  Loosely  described  it 
is  the  probability  that  at  the  end  of  the  planned  mission  the  system  is  still  in  operation  in  an 
acceptable  manner. 

Most  of  the  equipment  on  board,  apart  from  normal  control,  needs  service,  checkup, 
adjustments,' replacement  of  failing  units,  and  other  forms  of  maintenance  and  short-term 
repair  during  the  mission.  In  terms  of  reliability  theory,  materiel  and  personnel  are  "in  series": 
the  unavailability  of  one  invalidates  the  other.  The  mission  reliability  of  a  system  on  board  is 
the  joint  probability  that  at  the  end  of  the  mission  the  system  is  still  in  an  operable  state  or  on 
short  term  repairable  while  at  the  same  time  at  least  the  minimal  human  attendance  is  on  hand. 

By  way  of  precaution  against  nonoperation  of  an  entire  system  as  a  result  of  failure  of  a 
vital  part,  spare  parts  are  taken  along.  This  is  one  of  the  possible  forms  of  materiel  redun¬ 
dancy.  On  the  effect  of  materiel  redundancy  on  the  reliability,  a  reasonably  well-developed 
theory  is  available  [3].  In  [4],  the  composition  of  optimal  spare-part  kits  is  dealt  with.  Such  a 
kit,  of  all  kits  of  the  same  cost  ,  guarantees  the  highest  reliability  of  the  system  that  it  pertains 
to. 


In  the  situation  of  temporary  isolation  in  which  a  warship  finds  itself  during  a  mission, 
there  are  no  easy  means  to  replace  operators  and  maintenance  men  who  have  become  "hors  de 
combat."  Aboard  a  modern  and  automated  warship,  a  growing  percentage  of  the  crew  is  highly 
specialized  and  classified  in  a  considerable  variety  of  specializations,  which  makes  it  increasingly 
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difficult  for  one  to  stand  in  for  any  other  outside  his  own  category.  This  leads  to  the  necessity 
for  a  relatively  large  number  of  "spare"  crewmen  In  "expensive"  categories  to  be  taken  along. 
As  a  result  of  this  personnel  redundancy  it  may  be  found  that  a  high  degree  of  automation  does 
not  automatically  yield  the  savings  in  personnel  costs  that  are  generally  expected. 

In  view  of  these  tendencies  a  modern  management  demands,  within  operational  and 
financial  constraints,  a  careful  optimization  in  the  spending  of  funds  on  "spares,"  where  materiel 
and  personnel  aspects  are  not  separable.  This  paper  is  primarily  intended  to  be  a  contribution 
to  the  development  of  methods  for  the  modeling  and  solution  of  this  optimization  problem. 
Related  problems  were  treated  in  I5l . 

2.  A  MATHEMATICAL  MODEL  FOR  THE  SURVIVAL  OF  CREWMEN 

In  this  section  we  introduce  a  mathematical  model  for  the  probability  for  crewmen  to  sur¬ 
vive  a  mission. 

The  key  parameter  in  this  model  is  the  expected  number  of  hits,  r,  that  the  ships  incurs 
during  the  mission.  This  number  is  assumed  to  be  an  integer-valued  Poisson-distributed  ran¬ 
dom  variable  with  mean  g,  so 

(2.1)  Pit)  =  exp(-g)  • 


The  probability  for  one  particular  crewman  to  survive  a  hit  is  p.  In  consequence,  the  pro¬ 
bability  for  him  to  survive  /  hits  is  p',  and  the  probability  r  that  he  survives  the  mission  is 

(2.2)  r  =  £  p'Pii)  =  £  exp(-g)  •  -  exp(p  -  1)#. 

If  A  and  B  are  two  particular  crewmen,  then  at  each  hit,  their  survivals  are  assumed  to  be 
independent  events.  This  implies  that  the  probability  that  both  A  and  B  survive  t  hits  is  p^\ 
and  the  probability  r(2/2)  that  both  of  them  survive  the  mission  is 

(2.3)  r(2/2)  =  £  p''  Pit)  “  £  exp(-^f)  •  “  exp(p’  -  l)g. 

Because  of  their  common  dependence  on  /,  the  survivals  of  the  mission  by  A  and  by  B  are  no 
independent  events.  A  useful  measure  of  interdependence  is  the  (pairwise)  correlation 
coefficient  r,.„rr.  which  is  defined  in  the  following  way.  Consider  the  two  random  variables 
and  Xfl,  where  x.,  «=  I  if  /I  survives  the  mission  and  x^  -  0  if  he  does  not,  while  xg  is  defined 
similarly.  We  now  put 


C0V(X^.  Xg) 

Ivarix4)  •  var(x«) 


(2.5) 


EiXfXg)  -  Eix^}  ■  Eixg) 
Eix})  -  EHx^) 
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which  is  found  to  be 

o  f.)  r  exp(p-  -  1)«  -  exp2(p  -  l)g 

u.o;  „p(p- nj?-exp2(p- !)«■ 

If  Up  -  l)/fl  «  1,  we  can  use  the  approximations 

(2.7)  =  1  -  p  -  ^ 

g 

(2.8)  p  =  1  -  C„„  and  g  =  7^. 

^  corr 

If  ^fand  pare  varied  while  (1  -  p)pand  thus  /-remain  constant,  the  extreme  cases  are 
«  =  -  In  /•  and  p  =  0.  then  C\„„  -  I 
jf  —  00  and  p  — *  I :  then  C,„„  -  0. 

For  calculations  in  the  latter  (uncorrelated)  case,  the  following  property  is  useful 

(2.9)  exp(p'  -  Djf  —  exp/(p  -  1)^  «  r'  for  /  -  2.  3 . 


3.  A  IVfATHEMATICAL  MODEL  FOR  THE  MISSION  RELIABILITY 
OF  A  MANNED  SYSTEM 

For  the  case  where  materiel  and  personnel  are  "in  series,"  we  propose  an  extension  of  the 
mathematical  model  of  the  previous  section  as  will  be  demonstrated  by  the  following  example. 

We  are  dealing  with  a  system  that  must  be  manned  by  a  crew  of  minimally  r.  At  the 
beginning  of  the  mission,  n  crewmen  qualified  to  operate/maintain  the  system  are  on  board. 
During  the  mission,  the  ship  incurs  /  hits  where  i  is  integer- valued  Poisson-distributed  with 
mean  ^f.  At  each  hit,  an  individual  has  a  probability  of  survival  p,  while  the  materiel  of  the  sys¬ 
tem  has  a  probability  of  survival  a. 


Given  the  number  of  hits,  /,  the  probability  that  a  sufficient  number  of  crew  survives  is 
(see  appendix): 


(3.1) 


n 

n  : 

■ 

n  —  z 

z 

I  (-!)■' 

H-(l 

u 

z  +  u 

while  the  probability  that  the  materiel  survives  is 
(3.2)  o'. 

Hence,  the  probability  that  the  materiel  as  well  as  enough  personnel  survive  t  hits  is 


(3.3) 


n  «--■  ft  —  z  , 


The  mission  reliability  of  the  system,  which  is  the  probability  that  the  materiel  as  well 
as  enough  crew  survive  the  mission,  is  obtained  by  multiplying  (3.3)  by  P(r)  and  summing 
with  respect  to  t.  The  result  is 
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In  the  uncorrelated  case,  by  invoking  (2.9),  we  obtain  for  the  mission  reliability 


where  r^,  is  the  probability  that  an  individual  survives  the  mission  and  r„  is  the  probability  that 
the  materiel  survives  the  mission. 

A  Practical  Example 

With  r  =  1  we  have  for  n  =  1,  2,  3.  respectively,  in  the  correlated  case 
=  exp(ap  -  1)^ 

R^p  =  2  exp  (ay?  -  l)g  -  exp(ap’  -  1)« 

R^i,  =  3  exp  (ay;  -  Dy?  -  3  exp  (ay?*  -  l)jf  +  exp(ap'  -  l)g 
and  in  the  uncorrelated  case 

^ap  “  ^a^p 

n  ^  ^ 

^ap  *  ^^a^p  ^a^p 

Rap  *  ^^a^p  ^^a^p  ^a^p  ‘ 

We  vary  p  and  g  in  such  a  way  that  a 

ip  -  1  )g  -  constant  «  -0.0305 

keeping 

a  •  p 

which  means  that  the  probability  of  mission  survival  for  an  individual  as  well  as  for  the  materiel 
is  fixed  at 

r,  -  r„  -  0.970. 

In  Figure  1  we  plot  1  -  R^p  as  a  function  of  p.  In  order  to  show  the  influence  of  the  vul¬ 
nerability  of  the  materiel,  we  also  plot  I  -  Rp,  which  is  the  probability  that  enough  crewmen 
survive  the  mission,  regardless  of  materiel  survival.  The  quantity  Rp  derived  from  Rgp  by  put¬ 
ting  fl  -  1  or  Tfl  —  I . 

In  Table  1  we  list  I  -  Rp  and  I  -  as  functions  of  n  in  the  uncorrelated  case.  For 
n  —  oo  we  have  \  -  Rp  —  O  and  1  -  '  I  -  r^. 
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(D 

1  -  Rap 

It 

c 

u. 

O 

® 

idem 

for 

n  =  2 

idem 

for 

n=3 

® 

1-Rp 

for 

n  =  1  :  also  1  -  R 

1  -  Rp 

for 

n=2 

idem 

for 

n  =  3 

—  00 


TABLE  1.  Probabilities  of  Survival 
in  the  Uncorrelated  Case 


l-R, 

in 

in  % 

n  ■=  1 

3.00 

5.91 

w-  2 

0.0900 

3.09 

n  =  3 

0.00270 

3.00 

n—’oo 

0 

3.00 
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4.  THE  MISSION  RELIABILITY  OF  A  SYSTEM  COMPOSED  OF  TWO  SUBSYSTEMS 


We  consider  a  system  S  composed  of  two  subsystems  S,  and  S}.  For  the 
operation/ maintenance  of  S,  crewmembers  are  on  board  in  two  categories,  1  and  2.  Crewmen 
in  category  1  are  competent  to  operate/ maintain  (but  not  5;),  while  crewmen  in  category  2 
are  competent  to  operate/maintain  5^  (  but  not  S|).  S  cannot  be  in  operation  unless  Z|  crew¬ 
men  in  category  1  and  22  in  category  2  are  on  hand.  At  the  beginning  of  the  mission  there  are 
ni  crewmen  on  board  in  category  1  and  in  category  2.  The  ship  incurs  1  hits,  where  t  is  an 
integer-valued  Poisson-distributed  random  variable  with  mean  g.  At  each  hit,  the  probability  of 
survival  of  an  individual  is  p,  while  the  probability  of  survival  of  the  materiel  of  S|  and  Sj  is  Ui 
and  fli,  respectively. 


The  probability  that  after  /  hits  the  materiel  of  subsystem  survives,  along  with  a 
sufficient  number  of  crewmen  to  operate/ maintain  it,  is 


(4.1) 


M| 

n,  -  z, 

2l 

y  (-1)"'— ^ 

"1 

(see  appendix). 


For  S2  a  similar  expression  can  be  written.  After  multiplying  the  two  expressions  we  obtain  the 
following  expression  for  the  probability  that  the  materiel  of  both  subsystems,  along  with 
enough  crew,  survives  t  hits 


(4.2) 


Nil  "I 


2| 


2,  +  U^ 


£  (-l)"'^"' 
«2“0 


22 

22  +  «2 


rtj-zi 

«2 


(a, 02/2 


r,+i2+«l+U2)' 


By  multiplying  this  by  P(t)  and  summing  with  respect  to  r,  we  obtain  the  mission  reliabil¬ 


ity 

(4.3) 


- 


M| 

M|  -  2, 

1(^0 

Z2 

1 

N 

ry 

2| 

^2 

Ml 

22  +  M2 

1 

exp  (a\a;p'’*  -  l)g. 


In  the  uncorrelated  case  this  changes  into 


(4.4) 


M| 

M: 

2l 

22 

2,  +  M, 


Ml  -  2| 
Ml 


£  (-1)"'"'' 


II  .-0 


2i  +  U2 

where  and  are  the  probabilities  that  the  materiel  of  5,  and  S2,  respectively,  survives  the 
mission.  The  probability,  that  enough  crew  survives  the  mission  to  operate/ maintain  the 
two  subsystems,  irrespective  of  materiel  survival,  can  be  derived  from  by  putting  in  (4.3); 
fli  -  02  -  1  and  in  (4.4):  ”  r^j  -  1. 


M2  “  22 


r  r  r**  ■ 
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For  the  probability  /?„  that  the  materiel  of  both  subsystems  survives  the  mission,  we  have 
in  the  correlated  case 

(4.5)  /?„  =  exp  (fliflj  -  l)/f 

which  can  be  derived  in  the  same  fashion  as  (2.3),  and  which  in  the  uncorrelated  case  changes 
into 

(4.6)  R„  = 

5.  SOME  NUMERICAL  RESULTS 

The  quantities  R^„  R^^,  and  R„  as  dealt  with  in  the  preceding  section  were  calculated  for 
the  case  where 

zi  =  8  and  zj  *=  I 

while  p,  n^  and  were  varied;  O)  and  were  varied  along  with  psuch  that  always 

fll  =  At  ”=  p. 

As  p  was  varied,  p  was  varied  at  the  same  time,  keeping 
(p  -  Dif  =  constant. 

Three  runs  were  made,  with 


(p  -  Dp  =  In  0.990,  In  0.970  and  In  0.900, 

respectively. 

Samples  of  the  numerical  results  are  given  in  Tables  2,  3,  4  and  5,  and  in  Figures  2  and  3. 
In  Tables  2  and  3  we  list  1  -  R^  and  I  -  R^p,  respectively,  as  functions  of  (m),  ni),  as 
obtained  at  the  second  run,  in  the  uncorrelated  case.  In  Tables  4  and  5  we  do  the  same  for  the 
case  where  p  =  0.700.  In  Figure  2  we  plot  1  -  as  a  function  of  p  for  several  pairs  of  values 
of  (n\.  wj),  again  as  obtained  at  the  second  run.  The  corresponding  plots  of  1  -  R^p  and 
1  -  Rp  are  given  in  Figure  3. 

Tables  like  2,  3,  4  and  5  can  be  used  for  the  determination  of  optimal  sequences  of  alloca¬ 
tion.  Let  us  suppose  that  the  cost  per  crewman  per  mission  is  for  a  man  in  category  1  and 
k}  for  a  man  in  category  2.  Let  us  f^urther  suppose  that  we  can  afford  to  spend  an  amount  K  on 
the  crew  in  the  two  categories.  This  means  that  we  have  to  choose  ni  and  under  the  restric¬ 
tion  that 

(5.1)  Wikj  -t-  nikj  <  K. 

For  every  value  of  A,  it  is  easy  to  find  a  pair  of  values  (/ii,  ni)  such  that  under  the  res¬ 
triction  (5.1),  the  lowest  value  of  I  -  or  1  -  Rpp  is  obtained.  By  doing  this  for  increasing 
values  of  K,  the  optimal  sequence  of  allocation  is  established. 
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TABLE  2.  I  -  in  %  as  a  Junction  o/(n\,  nj),  for 
(p  -  1)^  “  In  0.970  and  p  —►1. 


«l 

■ 

2 

4 

8 

21.7 

21.6 

21.6 

9 

5.73 

2.82 

2.82 

10 

3.27 

il 

12 

3.00 

0.0916 

0.00432 

0.00169 

13 

3.00 

0.0900 

0.00280 

0.185  X  10"^ 

TABLE  3.  1  -  Rgp  in  %  as  a  Junction  of 
(«!.  wj)  for  (p  -  1)^  =  In  0.970  and  p  —  1 


1 

2 

3 

4 

8 

28.5 

26.3 

26.3 

26.3 

9 

11.3 

8.64 

8.56 

8.56 

10 

8.99 

6.25 

6.17 

6.17 

11 

8.75 

6.02 

5.93 

5.93 

12 

8.73 

6.00 

5.91 

5.91 

13 

8.73 

5.99 

5.91 

5.91 

14 

8.73 

5.99 

5.91 

5.91 

TABLE  4.  \  -  Rj,  in  %  as  a  function  of(n\,  /jj), 
for  (p  ~  l)g  —  In  0.970  and  p  —  0.7. 


1 

2 

3 

4 

8 

9.28 

9.17 

9.14 

9.13 

9 

8.39 

8.01 

7.90 

7.86 

10 

7.19 

6.44 

6.22 

6.15 

11 

5.97 

4.87 

4.53 

4.43 

12 

4.97 

3.55 

3.12 

2.99 

13 

4.23 

2.60 

2.10 

1.95 

14 

3.75 

1.96 

1.41 

1.24 

15 

3.45 

1.56 

0.985 

0.806 

16 

3.27 

1.33 

0.728 

0.542 

17 

3.16 

1.19 

0.576 

0.385 

18 

3.10 

1.10 

0.485 

0.290 

19 

3.05 

1.02 

0.394 

0.195 
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TABLE  5.  I-  in 
for  (p  -  l)g  - 


2 


%  as  a  Junction  of{n\,  nfi^ 
In  0.970  and p  —  0.7. 


1 

2 

— 

3 

B 

9.42 

8.85 

9.40 

8.80 

ggTil 

EBl 

8.4S 

8.09 

7.98 

7.94 

7.86 

7.32 

7.15 

7.10 

7.37 

6.68 

6.47 

6.41 

7.01 

6.22 

5.98 

5.91 

6.78 

5.92 

5.65 

5.57 

6.64 

5.73 

5.45 

5.37 

6.56 

5.62 

5.34 

5.25 

6.51 

5.56 

5.27 

5.18 

>^.2) 


M 


0  0.5  0.8  0.9  0.95  0.98  0.99  0.995  0.998  0.999 

- -  P 

Figure  2.  1  -  R,  vs.  p  for  several  values  of  («).  nj),  for  (p  -  1)  *  -  In  0.970 


VOL  28.  NO  3,  SEPTEMBER  1981 


NAVAL  RESEARCH  LOGISTICS  QUARTERLY 


'  A 


and 


512 


J.  K.  ZUIDWEG  AND  J.  VAN  DAM 


0  0.5  0.8  0.9  0.95  0.98  0.99  0.995  0.998  0.999 

- -  P 

Fioure  3.  I  -  Ata  and  1  -  vs.  p  for  several  values  of  (n|.  nj).  for  (/?  -  1)  «  -  In  0.970 

Results  are  depicted  in  Figures  4  and  S,  which  show  allocation  sequences  optimizing 
I  -  /?,,  and  1  -  respectively.  In  both  of  these  figures,  the  black  dots  refer  to  the  situation 
where 

k,  =  ki 

whereas  the  open  dots  apply  to  the  situation  where 

k,  ;  fe:  -  1  ;  2. 

In  the  former  case  the  sequence  is  discontinued  after  the  first  step  where  1  -  /?p  or 
1  -  Rap  is  reduced  by  less  than  O.S'Mi;  in  the  latter  case  the  sequence  is  continued  up  to  the 
same  final  value. 

The  effect  of  the  first  four  optimal  allocation  steps  (that  is  to  say,  optimal  for  the  case 
ilci  -  kj)  on  the  numerical  values  of  1  -  and  1  -  Rap  is  shown  in  Tables  6  and  7,  respec¬ 
tively.  The  parameters  in  these  tables  are  the  same  as  in  Figures  4  and  5.  In  Table  7  we  also 
give  the  limit  values  1  -  /?„  of  1  -  Rap  as  wj  and  nj  approach  infinity. 
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SOME  OBSERVATIONS  ON  THE  RESULTS: 

1.  Comparing  situations  where  p  is  equal  to  or  close  to  1  (so  ~  0)  with  situations 
where  p  is  smaller  (so  C^o,,  is  larger),  we  find  that  in  the  former  case  the  mission 
reliability  with  («),  M2)  “  (8.  D  is  smaller,  but  the  addition  of  spare  crewmen  is 
more  effectual. 

2.  Comparing  the  optimal  allocation  sequences  in  the  various  cases,  we  find  that  these 

sequences  are  not  too  strongly  dependent  on  p,  g,  fl|02  or  In  all  cases  m1/m2 

tends  to  become  smaller  than  Z\ii2-  In  other  words  we  recognize  a  tendency  to  allo¬ 
cate  a  relatively  large  portion  of  the  spares  to  the  smaller  number. 

3.  Comparing  the  allocation  sequences  for  a  particular  value  of  p,  while  In  (p  -  l)g  is 
varied,  we  recognize  the  tendency  of  n\ini  to  approach  zj/z:  as  exp  (p  -  l)g 
decreases:  if  the  risk  of  the  mission  increases,  the  extra  allocations  go  primarily  to 
the  larger  number. 

4.  There  is  a  significant  difference  between  spare  parts  in  a  spare  part  kit  and  "spare* 
crewmen  aboard  a  warship.  Only  parts  in  active  state  can  fail,  whereas  parts  in 
storage  ("cold  standby")  will  not  get  out  of  order.  A  "spare"  crewman,  on  the  other 
hand,  is  vulnerable  as  soon  as  the  mission  is  begun  (in  terms  of  reliability  theory,  he 
is  in  "hot  standby").  Despite  this  difference,  however,  the  results  obtained  here  are 
in  accordance  with  the  rules  which  were  found  to  apply  to  optimal  spare  part  kits  [4]. 

5.  Comparing  /?,,  with  /?<,/>  under  the  same  circumstances,  we  find  that  the  addition  of 
spare  crewmen  has  less  effect  on  Ra^  than  on  R^.  Since  the  difference  between 
and  Rp  is  caused  by  the  vulnerability  of  the  materiel,  this  illustrates  that  the  addition 
of  spare  crewmen  in  less  effectual  as  the  materiel  is  more  vulnerable.  Results  like 
the  ones  we  have  obtained  here  could  be  useful  for  making  the  choice  between 
spending  available  funds  on  spare  crew  or  on  improving  the  mission  reliability  of  the 
equipment. 

6.  CONCLUDING  REMARKS 

In  this  paper,  mathematical  models  for  the  mission  reliabilities  of  personnel,  and  person¬ 
nel  and  materiel  jointly,  were  presented.  An  example  was  worked  out,  where  the  optimal  allo¬ 
cation  of  crewmen  in  different  categories  of  specialization  was  calculated.  Despite  its  simplicity, 
this  example  is  indicative  of  the  results  that  can  be  obtained  by  application  of  the  same  tech¬ 
niques  to  more  complex  configurations. 

From  our  calculations  we  have  learned  that  the  mission  reliability  of  a  manned  system  not 
only  depends  on  the  probabilities  of  survival  of  individual  crewmen  and  separate  pieces  of 
equipment,  but  equally  on  the  correlations  between  survivals.  This  implies  that  efforts  must  be 
made  to  gain  insight  into  the  statistical  characteristics  of  risks  that  personnel  and  equipment  are 
exposed  to  during  a  mission. 
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APPENDIX 

Consider  n  possible  events  e\.  . e„  which  are  symmetrical  in  the  sense  that 

(A.la)  P(e,)  =  P(e,) P(e„) 

^A.lb)  P(e\e-j)  -  P(e\e^)  -=  —  «  P(e„  \  e„) 

(A.lc)  P(f’iCjf>,.)  “  Pieieje^)  - . -  P^e,,  :  i  c„) 


Then  the  following  property  is  known  (21.  The  probability  PO  z,  n)  that  at  least  revents 
happen  is 


(A.2) 


n 

n  -  : 

n  -  z 

P(>  z.  n)  = 

* 

1  (-1)" 

«-0 

u 

— 2 —  ir(z  +  u) 
z  +  u 


where  it(2  +  u)  is  the  probability  that  cj,  Cj .  <>.+„  all  happen,  irrespective  of  whether 

the  others  happen  or  not. 
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ABSTRACT 

The  stochastic  duel  is  extended  to  include  the  possibility  of  a  near-miss  on 
each  round  fired,  which  causes  the  opponent  to  displace.  During  displacement, 
the  displacing  contestant  cannot  return  the  fire  but  is  still  a  target  for  his  op¬ 
ponent.  An  alternative  interpretation  of  this  model  is  to  consider  the  displace¬ 
ment  time  as  the  time  a  contestant's  fire  is  suppressed  by  his  opponent's  fire 
and  that  he  does  not  move,  but  merely  ceases  fire  temporarily.  All  limes  are 
exponentially  distributed. 


1.  INTRODUCTION 

This  paper  continues  the  development  of  the  Theory  of  Stochastic  Duels.  Although  com¬ 
bat  is  a  very  complex  process,  microscopic  models  of  this  type  help  to  develop  important 
insights  into  the  more  complex  processes  and  are  also  useful  in  weapons  systems  analyses, 
among  other  things. 

Some  earlier  work  has  considered  mobility  by  incorporating  varying  projectile  time-of- 
flight,  see  Ancker  12]  and  Jaiswal  and  Bhashyam  [S],  or  by  varying  hit  probability,  see  Williams 
[8]  and  Bhashyam  and  Singh  [4]  and  Ancker  il].  Some  very  simple  models,  where  the  contes¬ 
tants  displace,  are  provided  in  Ancker  and  Williams  [3]  and  Schoderbeck  [6]  and  17].  In  this 
paper  we  consider  a  more  general  situation  of  the  latter  type. 

In  general,  most  earlier  studies  of  stochastic  duels  have  not  considered  an  interaction 
between  the  two  contestants,  A  and  B.  That  is  to  say,  the  same  results  would  have  been 
effected  if  each  Bred  separately  at  his  own  target  in  two  different  localities  and  they  subse¬ 
quently  compared  their  times  to  score  a  kill,  having  previously  agreed  that  the  quicker  of  the 
two  would  be  the  winner.  We  shall  now  consider  a  duel  where  i4's  behavior  is  contingent  upon 
Fs,  and  vice  versa. 


*This  study  was  partially  supported  by  the  Department  of  the  Army.  U.S.  Army  TRADOC  Systems  Analysis  Activity, 
White  Sands  Missile  Range,  New  Mexico. 
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2.  THE  MODEL 


We  assume,  as  usual,  that  both  duelists  begin  to  load  and  fire  simultaneously,  but  we  now 
add  the  possibility  that  one  or  the  other  may  score  a  near-miss,  the  effect  of  which  on  a  duelist 
is  to  make  him  move  to  a  new  firing  position.  We  may  imagine  that  if  he  did  not  move,  he 
would  immediately  be  killed,  since  his  opponent  has  now  gotten  his  range.  During  his  displace¬ 
ment  time,  he  is  subject  to  fire  from  his  opponent  but  cannot  return  it.  We  assume  that  the 
probabilities  of  a  near-miss  and  a  kill  are  the  same  from  round  to  round,  and  that  once  a  duelist 
has  displaced  because  of  a  near-miss,  he  proceeds  to  load  and  fire  as  before.  His  displacement 
and  firing  times  are  random  variables  whose  probability  density  functions  are  known  and  are 
not  necessarily  the  same. 

An  alternate  interpretation  that  may  be  useful  is  as  follows:  Upon  receiving  a  near-miss, 
the  duelist  merely  seeks  cover  in  his  present  position  and  ceases  fire  for  a  period  of  time  equal 
to  the  corresponding  displacement  time,  i.e.,  merely  interpret  displacement  time  as  fire- 
suppression  time,  and  the  model  is  a  fire-suppression  model  as  it  stands. 

3.  ANALYSIS 


Our  first  step  is  to  eliminate  from  consideration  all  complete  misses  on  both  sides.  We 
are  thus  left  with  a  series  of  near-misses  that  form,  so  to  speak,  a  succession  of  turning  points 
on  which  the  duel  hinges.  It  does  not  matter  which  of  the  duelists  scores  a  near-miss,  since  it 
still  interrupts  the  course  of  the  duel.  We  thus  make  a  list  of  when  the  near-misses  occurred 
and  who  scored  them.  Ultimately,  the  duel  ends  on  a  near-miss  which  was  actually  a  kill. 


We  introduce  the  following  notation.  Let 

PJ^^  •>  the  conditional  probability  that  A  scores  the 

next  near-miss,  given  that  A  scored  the  last  one. 


(1) 


Pba  ~  ific  conditional  probability  that  B  scores  the 

next  near-miss,  given  that  A  scored  the  last  one, 
and  similarly  for  and  Psp.  Then,  we  plainly  have 


(2) 

Next,  let 

(3) 

and 


Paa  +  Pba  “  Pab  +  Pbb  “  1  • 

p^  -  the  unconditional  probability  that  A  scores 
a  near-miss  on  any  round. 


k4  -  the  conditional  probability  that  A  scores 
a  kill,  given  a  near-miss, 
that  is,  on  any  near-miss,  a  kill  may  or  may  not  have  occurred,  and 
I^aPa  ~  joint  probability  of  a  near-miss  and  a 
kill,  or  just  simply  a  kill. 


(4) 


(5) 
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Also,  lei 

(6) 

and 

(7) 


<Ia  “  ^  ~  Pa  the  probability  that  A  misses 
entirely  on  any  round 

/^  -  1  -  -  the  conditional  probability  that  A  does 

not  score  a  kill,  given  a  near-miss. 


Of  course,  there  is  a  similar  notation  for  B.  We  may  remark  here,  that  it  will  be  important  in 
some  future  research  to  allow  these  probabilities,  i.e.,  (3),  (4),  (6)  and  (7),  to  be  different  in 
the  suppressed  and  nonsuppressed  states. 


We  further  let 


(8) 


Po4  =  the  probability  that  A  scores  the  first  near-miss 
Pag  ^  the  probability  that  B  scores  the  first  near-miss 


so  that 


(9) 

and  we  call 

(10) 


POA  +  POH  “  i 

\P,SA)  =  the  probability  that  A  kills,  for  the 
first  time,  on  near-miss  number  n 


PJB)  =  the  probability  that  B  kills,  for  the 
first  time,  on  near-miss  number  n 
Then,  the  probability,  P(A),  that  A  wins  the  duel  is 


(11) 


/’(/I)  -  £  /»„(/<)  , 
and,  similarly  for  B 

P(B)  -  £  P„{B)  . 


Also,  P(A)  +  PiB)  -  1.  Clearly,  when  n  -  I,  we  have 
(12)  P)iA)~p„^k,  and  P^iB)  •  Poaka- 


Now.  the  probability  that  A  scored  a  near  miss  on  round  n  —  1,  but  did  not  kill,  is  given 
by 

since  P„-tiA)  includes  a  factor  for  the  kill  on  round  n  -  1.  which  factor  must  be  replaced 
by  /.)  if  there  was  no  kill.  Thus, 
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(13)  P„(A)  =  k,  p,,  P„  ,(/()  +  p^„  P„  ,(B) 

provides  the  two  near-miss  situations  on  a  given  round  that  can  lead  to  a  kill  by  A  on  the  next 
round.  Hence, 


and  similarly. 


/’„(/!)  =  l^p^^P,,  t<A)  +  l„  P4„  —  P„-,iB). 

«« 


P.SB)  -  /«  p«8  P„^,(B)  +  I,  p„,  P„^M). 

*4 


Summing  both  sides  of  these  simultaneous  difTerence  equations,  from  2  to  «>,  and  making 
use  of  Equations  (11)  and  (12),  we  find  that 

(16)  PU)  -  p(uk,  =  I,  p^i  PU)  -I-  /«  PiB  PiB), 


(17)  P(B)  -  Poakg  —  Ig  Pgg  piB)  4-  pg^  —  P(A). 

A-( 

When  we  solve  these  two  equations  simultaneously  for  PiA),  we  obtain 

(18)  PU)  -  ^  . 

kn  ~  Ia  Paa  kg  +  Ig  p^g  k^ 

with  a  corresponding  expression  for  P(B).  By  using  Notations  (1),  (2),  and  (7),  it  is  easily 
shown  that  the  denominators  of  P(A)  and  P(B)  are  equal. 

Now,  adding  the  numerator  of  P(A)  to  that  of  P(B),  we  find 

kA  kgPo^  -t-  k4  Ig  p^g  +  kg  P(,g  +  kg  I  a  PgA , 

which,  by  using  Equations  (2)  and  (9),  is  equal  to 

kA  kg  +  k^  Ig  Pah  A-  kg  Ia  —  kg  Ia  Paa  • 

And  now,  using  Equation  (7),  this  is  equal  to 
ka  +  kA  Ig  Pah  ~  kg  Ia  Paa  > 

which  is  equal  to  the  common  denominator  of  P{A)  and  P(fl),  thus  checking  that  PiA)  -E 
P(B)  -  1. 

Now,  assume  that  ^'s  firing  time  is  exponentially  distributed  with  mean  1/r^  (r^  is  >4's 
rate  of  fire),  and  his  displacement  time  is  also  an  exponential  with  mean  8^,  and  similarly  for 
B.  Then,  by  exactly  the  same  reasoning  as  used  in  the  fundamental  duel,  [9],  to  arrive  at  the 
first  kill,  t  he  probability  that  A  makes  the  first  near-miss,  Pqa,  is 

/ 1  nA  Pa  ’’a 

()9)  PoA  ”  _  ,  i  _  ,  • 

PAf^A  +  Pbi^b 

Also,  from  Notation  (1), 

(20)  Paa  -  P\Ta  <  Tg  +  Dg] 
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where 


=  random  variable  time  between  A ’s  near-misses, 
Tg  =  lime  between  B's  near-misses,  and 
Dg  =  0's  lime  to  displace. 


This  expression  accounts  for  the  fact  that  after  A  scores  a  near-miss,  he  proceeds  to  fire 
again,  whereas  0  must  first  displace  and  then  fire.  And,  as  we  have  already  seen,  when  the 
probability  of  the  event  (near-miss)  is  and  the  firing  lime  is  exponential  with  mean  1/r^, 
then  the  time  to  score  a  near-miss  is  exponentially  distributed  with  mean  XIpaI'a-  Again,  we 
may  use  the  solution  of  the  fundamental  duel  to  obtain  this  probability,  Paa^  by  noting  that  the 
characteristic  function  of  Tg  +  Dg  is  simply  the  product  of  their  individual  characteristic  func¬ 
tions.  Hence, 

^  J_  r“-'‘  _ PAriPurgdu _ 

liri  i«  (p^r^  -I-  iu)  (pgtg  —  iu)  (1  -  i&gu) 
and,  using  residue  theory,  we  have 

07)  =  +  PirA^B  +  ParB^a) 

(p^ri  +  Pb^h)  (•  +  PArA  6«)  ’ 

and  similarly. 


(23) 


_ _ ^ _ 

^Pa'‘4  +  /»«'•«)  <•  +  Parn^A^  ' 


When  we  substitute  these  results  into  Equation  (18),  we  obtain,  after  a  little  algebra, 


(24) 


fe^P^r4(^  4-  (1  -I-  kgpgrghA) 

kAPAfA^X  -f-  p^r^Ba)  (1  +  kgpgtghA) 
+  kgpgrg{\  -E  PgrghA)  (1  -E  /c^p^r^b^) 


4.  CONCLUSION 

It  is  interesting  to  note  that  if  -=  lc«  -  1,  then  all  near-misses  are  really  kills,  and  Equa¬ 
tion  (24)  reduces  to  the  solution  for  the  fundamental  duel  with  exponential  firing  times,  l9l, 
and  the  same  result  occurs  if  84  =  fig  —  0  (zero  displacement),  as  it  should. 

The  expression  in  Equation  (24)  is  unwieldy,  since  it  contains  eight  parameters.  If  we 
define 

Ph  “  Ph'h^h> 

0^-p.,r^8^.  and 
D  -  hgih  , 

then  Equation  (24)  becomes 

Dk,  Pa(D  -E  kgPg)  (I  -E  DPa) 


(25) 


Pi  A) 


kADP^iD  -E  kgPg)i\  -E  DPa)  -E  kgPgiD  -E  Pg)  (1  -E  KaDPa) 
which  reduces  the  parameters  to  only  five. 
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